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ABSTRACT 


Today’s first responders confront a common challenge, namely the lack of 
exposure to and experience with asymmetric threats (i.e., terrorism and natural disasters) 
in training venues that would enable them to develop familiarity with these novel 
situations. Different problem-solving strategies currently employed by today’s first 
responders are described, along with situation awareness and how to best leverage first- 
responder experience. Literature on expert versus novice decision making, situation 
awareness, recognition-primed decision making, and scenario-based learning was 
leveraged to design the thesis experiment. Through scenario-based exercises, the thesis 
attempted to discover whether the decision-making skills of an experienced fire officer 
(expert) can be learned by newly promoted officers (novice). Results from this 
experiment provided insight and plausible remedies regarding today’s asymmetric threats 
in the form of recommendations to enhance the first responder’s ability to develop good 
situational awareness and decision making 

The goal now is to use research results and recommendations as a springboard to 
develop training that helps a novice to effectively respond to asymmetric threats. 
Experiment results indicate that, by combining scenarios designed to expose novices to 
situations they may not experience during routine operations with timely expert feedback, 
an individual’s decision-making skills and situation awareness can be improved. 


v 



THIS PAGE INTENTIONALLY LEFT BLANK 


vi 



TABLE OF CONTENTS 


I. INTRODUCTION.1 

A. PROBLEM STATEMENT.2 

B. RESEARCH QUESTION.6 

C. LITERATURE REVIEW.7 

1. Asymmetric Threats (Terrorism/Natural Disaster).7 

2. Decision Making.14 

3. Situational Awareness.18 

4. Expert vs. Novice: Characteristics of Expert Decision Making ....19 

5. Rational Choice and Naturalistic Decision Making.24 

6. Computer-Based Simulation Technologies.27 

7. Decision Skills Training.29 

8. Literature Summary.31 

D. HYPOTHESIS.31 

E METHODOLOGY.34 

F. SUMMARY.35 

II. SITUATIONAL AWARENESS.37 

A. LEVEL 1 SA: PERCEPTION.39 

B. LEVEL 2 SA: COMPREHENSION.40 

C. LEVEL 3 SA: PROJECTION.41 

1. Limited Attention and Working Memory.42 

2. Default Values.43 

3. Automatic Action Selection.44 

4. Categorization of Information.45 

5. Data-Driven and Goal-Driven Processing.46 

6. Expectations.47 

III. NATURALISTIC DECISION MAKING.49 

A. CUE RECOGNITION.53 

B. PATTERN MATCHING.58 

C. MENTAL MODELS.61 

D. MENTAL SIMULATION:.67 

IV. DECISION-MAKING STRATAGEM.73 

A WHAT TO TRAIN (SKILLS).74 

B. HOW TO TRAIN (METHODS).75 

V. SCENARIO-BASED LEARNING AND EXPERIMENTAL DESIGN.77 

A. EXPERIMENT HYPOTHESIS.78 

B. GROUP DESIGN.79 

C. PRE-EXPERIMENT QUESTIONNAIRE.79 

D. THESIS EXPERIMENT.80 

E. EXPERIMENTAL / CONTROL GROUP SCORING.84 

F. SUMMARY.86 

vii 










































VI. ANALYSIS AND RECOMMENDATIONS.89 

A. RECOMMENDATION.89 

APPENDIX A.95 

APPENDIX B.97 

APPENDIX C.99 

APPENDIX D.103 

APPENDIX E.113 

APPENDIX F.119 

APPENDIX G.129 

APPENDIX H.131 

APPENDIX 1.133 

APPENDIX J.135 

BIBLIOGRAPHY.137 

INITIAL DISTRIBUTION LIST.145 


viii 

















LIST OF FIGURES 


Figure 1. Seven-Step Problem Solving Model.5 

Figure 2. Conditions Favoring Intuitive and Analytical Approaches.25 

Figure 3. Automaticity .45 

Figure 4. Recognition-Primed Decision Model.51 

Figure 5. Expert Group Experiment Matrix.82 

Figure 6. Experiment / Control Group Experiment Matrix.83 









THIS PAGE INTENTIONALLY LEFT BLANK 


x 



LIST OF TABLES 


Table 1. Comparison of Expert and Novice Decision-Making Characteristics.21 

Table 2. Skills to be Trained and Methods.24 

Table 3. Simple House Fire.67 

Table 4. NDM Characteristic and Mechanism/Expertise.74 

Table 5. Sample Earthquake Scenario Passage.85 








THIS PAGE INTENTIONALLY LEFT BLANK 



ACKNOWLEDGMENTS 


There are many individuals I would like to thank for their support and 
contributions in making the completion of this Thesis and Master’s Program a reality: 


Thank you to the FDNY Commissioner Nicholas Scoppetta, FDNY Chief of 
Department Salvatore Cassano, and FDNY Chief of Operations Patrick McNally for 
allowing me to attend the Naval Postgraduate School (NPS). Tha nk you, also, to Chiefs 
Michael Weinlein, Joseph Pfeifer, Michael Puzziferri, and Steve Raynis for encouraging 
me to follow in their footsteps at NPS. 


To the staffs at the FDNY Center for Terrorism and Disaster Preparedness for 
providing invaluable assistance, insight, and feedback, beginning with my application 
process and continuing through my thesis research. And, to the FDNY First Line 
Supervisors Training Program for assisting in my thesis experiment, thank you. To 
Robert Burns FDNY and Wayne Trigg for their advice and counsel, influencing my 
paradigm of what leading from the front really means. To those fire officers from the 
FDNY and neighboring fire departments throughout New York State who took time out 
of their busy schedule to help make my research experiment a success, thank you. 


A special thanks to my thesis advisors Lauren Wollman and Susan Hutchins for 
believing in my idea and providing me the opportunity to succeed. Your experience, 
wisdom, and counsel were invaluable to my research efforts and final product. To 
Richard Bergin and Erika Schaub for their assistance in my research experiment analysis, 
and all the faculty and staff at the Naval Postgraduate School for providing never-ending 
assistances during the past eighteen months. To Cohorts 0603/0604, thank you for 
making our time together a rewarding experience. To Mr. Airpower, my walking partner, 
you are the real deal! 

xiii 



To my loving wife, Terry, whose patience, support, and understanding throughout 
the past thirty years, always encouraging me to take the road less traveled when I wanted 
to follow the pack, thank you. You are my best friend. To my loving daughters Jillian, 
Kimberly, and Elyse, three reasons for making this world a safer place to live, thank you. 


To all first responders who make supreme sacrifices each and every day by 
placing themselves between us and the natural disasters and acts of terror that 
mean to do us harm, thank you. 


xiv 



I. INTRODUCTION 


Naturalistic decision making is the way individuals use their experience to solve 
problems in real-world settings, identifying their situation and taking action during times 
of uncertainty. Decision-making theorist Gary Klein believes that today’s first responders 
often use their “power of intuition” when problem solving in their respective lines of 
work. They have to — time is a rare commodity at a fire, in an emergency room, or at a 
crime scene. Experienced firefighters, nurses, and police officers exhibit expert problem¬ 
solving skills every day in their emergency response roles. 

Thesis research — and beliefs regarding naturalistic decision making, 
expert/novice problem solving, and situation awareness — remains loyal to those of 
Klein, Endsley, and Cohen. Namely, that effective first-responder problem solving is 
often a product of experience. While first responders have been interviewed and studied 
regarding their decisions making experience and processes employed, this thesis connects 
decision-making theory with current novel real-world problem scenarios by identifying 
the problems facing today’s first responders, plausible solutions, and recommended 
treatments for training personnel on novel situations. 

This research was conducted for several reasons. First, it joins the beliefs of 
today’s leading decision-making theorist, Gary Klein with first-responder problem 
solving via real-world examples and scenarios. Second, through the use of real-world 
examples derived from personal experiences and shared story telling, an individual often 
sees the relevance of the theory to their particular domain. Third, when the connection 
between theory and reality is discovered, the learning has meaning. For instance, what 
does an expert consider and what analogues does he create when solving present-day 
problems? These considerations and analogues are captured through this thesis’ scenario- 
based examples and experiments. 


An expert and novice fire officer may intuitively agree on the risk vs. reward 
factors regarding the commitment of firefighting personnel to a vacant building fire. 
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When problems become complex, however, resulting in decision-making situations that 
require more than basic knowledge to solve, individuals must rely on more advanced 
skills to help them reach a decision. For example, when the officer in command receives 
reports that the building is now in danger of falling down and that squatters have been 
living in the building, the risk vs. reward factors become more complex. 


By providing exposure to the skilled performance that reflects experts’ intuitive 
thought processes during everyday emergencies, today’s less-experienced decision 
makers can begin acquiring the same skills through structured scenario-based training. 
When expert feedback on their responses to asymmetric scenarios is provided to novices, 
the novices can begin to leam how to think through a problem in a way that is similar to 
the experts. Only then can one attempt to take those expert skills and talents and try to 
superimpose them onto a novice’s problem-solving skills in an asymmetric environment. 

A. PROBLEM STATEMENT 

Fire service personnel solve fire ground problems using several decision-making 
strategies; one approach employs the recognition-primed decision-making (RPD) 
strategy. RPD includes several techniques: pattern matching, cue recognition, mental 
simulations, and mental models. Pattern matching helps fire personnel recognize 
similarities to other situations by matching cues from the current situation such as the 
color of smoke with templates stored in memory and finding a match. A template is a 
group of cues, stored in long-term memory that represents a particular instance of a 
situation, for example, a specific type of fire. Cue recognitions help the decision maker to 
identity critical pieces in the patterns. Mental simulation allows the decision maker to 
mentally simulate applying a particular course of action to see how well it will work. 1 
Mental models store information in an organized fashion, and are often referred to as the 
decision maker’s “schema.” 2 Mental models afford decision makers the ability to 

1 Gary Klein, Sources of Power: How People Make Decisions (Cambridge, Mass: M.I.T. Press, 1998), 
92. 

2 Mica R. Endsley, “Expertise and Situation Awareness” (eds) Neil Chamess et al., The Cambridge 
Handbook of Expertise and Expert Performance (New York: Cambridge University Press, 2006): 638. 
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construct personal paradigms (the individual lens we use to view the world) that can 
accommodate the current dynamic event. Many first responder decision-making 
techniques are learned through trial-and-error problem solving during emergency 
response events. Once these techniques are mastered, fire service personnel use them as 
decision-making tools to effectively assist them in their problem-solving strategies during 
an emergency response operation. 

A second strategy uses fire service Standard Operating Procedures (SOPs). SOPs 
are developed from fire service training guides and related publication manuals. The 
procedures included in training guides are the result of over one hundred years of 
collective on-the-job experience, and are reinforced through training and exercises. For 
example, an effective method to evaluate a fire department’s SOPs and operational 
readiness concerning a high-rise building fire is to conduct a full-scale exercise in a 
building of this type. From a decision-making standpoint, SOPs are imperative because 
fire service personnel from a variety of locals often respond to the same incident; to work 
together effectively, they need common points of reference and similar training 
backgrounds. The National Incident Management System (NIMS) addressed multi¬ 
department operations by introducing an incident management system that provides a 
consistent approach for all levels of response to work together in preparing for, 
responding to, and recovering from domestic incidents regardless of size or complexity. 3 
New York City’s Citywide Incident Management System (CIMS) is the City’s 
implementation of NIMS for managing emergency incidents and planned events in New 
York City. 4 For example, an incident commander’s operational decision needs to be 
understood, not only at that strategic level, but also at the tactical level. Because strategy 
and tactics are somewhat streamlined throughout the fire service, decisions made and 
problems solved will be predicated on the common goal of protecting life and saving 
property. 


3 Department of Homeland Security, National Incident Management System (March 1, 2004): 1. 

4 New York City Office of Emergency Management, Citywide Incident Management System (April 6, 
2005): 6. 
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In their attempt to solve problems and make effective decisions under emergency 
conditions, fire service personnel synthesize their own real-life firefighting expertise with 
the collective experiences of numerous fire service personnel conveyed via SOPs, 
training, and education. Experience, developed through direct participation and 
observation, has given the fire service a distinct advantage when it is required to solve 
problems in their area of domain-specificity. 

In addition to their everyday problem-solving and decision-making responsibilities, 
fire service personnel are now confronting a novel concern: decision making in an 
asymmetric threat environment, or situations with environmental factors that are unorthodox, 
surprising, urgent, and unforeseen. The decision strategies normally employed by fire 
personnel are not always effective because these novel situations do not match the stored 
patterns the firefighter has accumulated from years of experience. Pattern matching, cue 
recognition, and mental simulations do not always apply to these unique situations. Examples 
of an asymmetric threat environment include chemical, radiological, biological, and other 
forms of terrorist attacks on critical infrastructure (such as 9/11), and natural disasters 
(wildland fires, hurricanes, earthquakes, and floods) that threaten a population that has grown 
in density over the past fifty years, and thus are of much greater consequence than those for 
which fire services were traditionally trained and prepared. Urbanization of the rural 
landscape has changed the way forest fires are fought. Trees are no longer the resources most 
severely exposed to fire, now it is the homes and the people living in the forest.5 As James 
Davis has noted, ‘Tire managers across the nation are confronting the rapidly developing 
problem of people moving into wildland areas, increasing what has been termed the 

wildland-urban interface.”^ 

Most significant fire-related events have the potential to generate many casualties 
and cause widespread environmental damage. An asymmetric attack is not inherently 
worse, but is rather unconventional in some aspect, obliging responders to operate in a 
world of uncertainty. The fire service now confronts a new threat without the benefit of 

^ WUI Professional Development Program, “Module 1: Trainer’s Guide Wildland-Urban Interface 
Issues and Connections.” http://www.interfacesouth.org/products/pdf/modl.pdf . [Accessed June 8, 2007]. 

6 James B. Davis, “The Wild land-Urban Interface: What It Is, Where It Is, And Its Fire Management 
Problems” (n.d.): 160. http://www.cnr.berkelev.edu/~fbeall/050.pdf . [Accessed June 8, 2007]. 
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the real-life experience and the SOPs necessary to create effective situational awareness 
(SA). “Simply put, SA is knowing what is going on around you. Inherent in this 
definition is a notion of what is important.” 7 


Expert decision makers are said to think and act intuitively, often relying on “gut 
instinct” — honed by experience — to make crucial decisions. This is naturalistic decision 
making in its most basic form. Without experience, though, decision makers may reason like 
novices. Because they have no mental models stored (based on experience), there are no 
patterns to match or cues to recognize. In some cases, the decision maker relies on a rational 
choice strategy. These strategies may not produce the best course of action (COA) for first 
responder decision makers to use in emergency situations. The problem with rational choice 
strategies is that, because they involve methodical, multi-step processes, they are labor-and 
time-intensive; because of this, the decisions are often obsolete and ineffective when it comes 
time to institute them. The following seven-step problem-solving model from William 
Duggan’s “Coup D’oeil; Strategic Intuition in Army Planning.” is one such example of this 
lengthy process. 
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1 - ID the problem 


2 - Gather information 


3 - Develop criteria 


4 - Generate possible solutions 


5 - Analyze possible solutions 


6 - Compare possible solutions 


7 - Make and implement the decision 


Who, what, when, where and why? 


Facts, assumptions, and interests 

J LINKED J 

Screening & Evaluation 


Suitable, feasible, acceptable, 
distinguishable, and complete 


Benchmark {Does the solution achieve 
the desired state?) 


Determine the best solution 


Decide and act 


Figure 1. Seven-Step Problem Solving Model (from 8 ) 


7 Endsley “Theoretical Underpinnings of Situational Awareness: A Critical Review Situation 
Awareness Analysis and Measurement (Mahwah, N.J.:Lawrence Erlbaum Associates, 2000): 2. 
http://www.satechnologies.com/papers/pdf/satheorychapter.pdf [Accessed April 2007]. 

8 William Duggan, “Coup D ’oeil: Strategic Intuition in Army Planning” (November 2005): 15. 
http://www.StrategicStudiesInstitute.army.mil/ [Accessed January 3, 2007]. 
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By relying on a rational thought process, Klein suggests that inexperienced 
decision makers are viewing the problem via a very specific narrow lens. Klein, a leader 
in the study of problem solving and decision making, calls this narrow vision approach 
“hyperrationality,” because decision makers try to solve all problems based on logical 
and analytical forms of reasoning. 9 Decision makers try to do all their thinking just by 
applying a rational procedure. Imagine the ramifications of incorporating a narrow lens 
approach while addressing a homeland security prevention, preparedness, or response 
related matter and not seeing the application process in its entirety. Envision a homeland 
security prevention initiative that was not based on experience in the decision-making 
process. Prevention planners use past events as starting points for future plans. By not 
utilizing their previous experience as a foundation for making decisions, planners are at 
risk of missing the “big picture.” 

Many real-world events do not allow for an in-depth analysis and size-up of the 
scene. A fire officer operating at an expanding fire using a rational choice strategy would 
take so much time to finally decide on ordering an additional alarm response that the 
level of response would no longer be adequate due to the speed of the spreading fire. 
Instead, when responding to these events, firefighters frequently rely on spontaneous 
decision-making and problem-solving skills based on many years of accumulated 
experience and training that has enabled them to develop expertise. When operating in an 
environment that demands immediate action, use of the more analytic rational choice 
strategy would require too much time to be effective. Because many events are so 
dynamic, well thought-out decision-making strategies may not be applicable when it 
comes time to institute them. 

B. RESEARCH QUESTION 

The research question investigated for this master’s thesis is: How to improve first 
responder problem-solving and decision-making skills in today’s new threat 
environment? 


9 Klein, Sources, 260. 
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C. LITERATURE REVIEW 

The literature discussed in this chapter concerning first responder problem solving 
and decision making is divided into three categories: the asymmetrical environment, 
decision-making strategies, and scenario-based training. 

First, is the asymmetric environment that first responders are currently 
experiencing, including both natural and man-made disasters. Experts in the field of 
terrorism and disaster preparedness (Hoffman, Falkenrath, and Linstrom) provide 
commentary on the current state of readiness and perceived threat. Discussions regarding 
first responder levels of preparedness, along with recommendations designed to enhance 
individual awareness levels regarding terrorism (McKinsey Report) and natural disaster 
readiness (Federal Response Plan to Hurricane Katrina), underscore the decision-making 
process. 

Second, are decision-making strategies and theoretical frameworks, especially as 
they pertain to expert versus novice decision making and to rational choice versus 
recognition-primed decision strategies. How applicable are the fire/emergency decision¬ 
making strategies at a hurricane or radiological attack? How does an expert prioritize 
decision making? Klein, Duggan, and Endsley weave intuition, rational choice, and 
situation awareness into a montage of decision-making analogues. 

Third is scenario-based training in the first responder decision-making equation. 
Scenario-based training can provide effective decision-making training without the 
expense and risk of full-scale types of exercises, and can be self-paced to accommodate 
various levels of an individual’s level of expertise and competence. Scenario-based 
training helps practitioners connect theory with real-world applications/situations. A 
scenario-based experiment (paper and pencil) comparing first responder experts and 
novices was the focus of this thesis research. 

1. Asymmetric Threats (Terrorism/Natural Disaster) 

In his paper, Problems of Preparedness: U.S. Readiness for a Domestic Terrorist 
Attack, Richard A. Falkenrath, the Deputy Commissioner of Counterterrorism in the New 
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York Police Department, offers a suggestion to reduce the uncertainties of response. 
Citing chemical and biological weapons as examples of asymmetric threats (uncertainty), 
Falkenrath believes that first responders need to develop a deeper understanding of these 
weapons (i.e., the agent’s properties of dissemination and interaction with the 
environment). Additional considerations include the effects of an asymmetric attack upon 
the civilian population, and how the civilian reaction will affect response plans. Will they 
become part of the problem? Falkenrath suggests that the public’s reaction to a weapon of 
mass destruction (WMD) incident needs to be integrated into the response plan. 10 For 
example, a radiological “dirty bomb” attack may require a mass/gross (copious amounts 
of water) or technical (individual) decontamination of those exposed before medical 
triage, treatment, and transport operations can begin. The public will view 
decontamination as necessary or superfluous (depending on the degree of contamination), 
especially when medical treatment is vital. How do first responders corral possibly 
contaminated individuals so they can be assessed before they drift from the scene? 
Whereas gross decontamination is designed to decontaminate large numbers of people by 
passing them through a water shower/curtain often provided by a hose line, this 
procedure might not be practical or well received in colder temperatures. How much 
experience do first responders possess regarding the public’s compliance/cooperation 
with these essential procedures? 

In his article, “Weapons of mass destruction a threat from desperate al-Qaeda, ” 
Tom Allard reports that, according to Bruce Hoffman, an expert in the study of terrorism- 
suicide bombers, the possibility of a terrorist attack using chemical, biological and 
radiological weapons is growing. These weapons have not only a corrosive (physical) 
effect but also a psychological one. Hoffman believes that the most likely unconventional 
attack would come from a radiological or “dirty bomb.” “In a city like New York, the 


10 Richard A. Falkenrath, “Problem of Preparedness: U.S. Readiness for Domestic Terrorists Attacks,” 
International Security, 4.(MIT Press: Spring 2001): 13. 

http://www.uky.edu/RGS7Patterson/desch/Readings/02-13/02-13_falkenrath_l .html. [Accessed May 23, 
2007]. 
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economic impact would be immense.” 11 This type of device causes more panic than 
damage, while undennining public confidence in the government. Hoffman’s concerns 
regarding public reaction reinforce Falkenrath’s suggestions to incorporate public 
reaction into the response plan. Hoffman further cites the recent chlorine bomb 
explosions in Iraq as another “unconventional” weapon that caused minimal property 
damage while resulting in panic: “There are an endless number of truck bombs that have 
killed in the hundreds but look at the reaction of Iraqis to chlorine bombs. They are 
absolutely panicked. ” Hoffman feels that the terrorists are more dangerous than before 
the September 11 attacks because they are more desperate. 12 

In Psychology of Terrorism, Richard Gist identifies the inherent resiliency of 
today’s first responders when it comes to post-traumatic disorders from the nature and 
emotional impact of their work. Several reports regarding post-traumatic stress disorder 
(PTSD) reveal that first responders are “uncommonly resilient.” 13 This resiliency enables 
firefighters to provide a social function that must be exercised for communities to 
prosper. 14 While fire organizations are their communities’ first resources, providing the 
nonnal functions related to hazards and safety, many fire organizations now require the 
expertise regarding hazardous materials identification and mitigation, mass casualty 
triage, decontamination, and technical rescues from natural disaster and structural 
collapse. 15 Acquisition of this expertise is correlated with decision making in two distinct 
ways. First, the recognition of first responder resiliency, understanding that firefighters 
do not get paid for what they do, but rather for what they might have to do, makes it 


1 Horn Allard, “Weapons of mass destruction a threat from desperate al-Qaeda,” Sydney Morning 
Herald (May 24, 2007). http://www.smh.com.au/news/world/weapons-of-mass-destruction-a-threat-from- 
desperate-alqaeda/2007/05/23/1179601492715.html. [Accessed May 5, 2007]. n.p. 

12 Ibid. 

13 Richard Gist, “Promoting Resilience and Recovery in First Responders,” (eds) Bruce Bongar, Lisa 
M. Brown, Larry E. Beutler, James Breckenridge, Philip G. Zimbardo, Psychology’ of Terrorism (New 
York: Oxford University Press, 2007): 418. 

14 Ibid., 419. 

15 Gist, “Promoting Resilience and Recovery in First Responders,” 418. 
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imperative that decision making is at an expert level. Second is the recognition that 
additional expertise related to natural disasters preparedness is in accord with current 
asymmetrical threat planning. 

John Linstrom retired at the rank of assistant chief after twenty years of service in 
fire departments in Texas and California; presently, he is an educational consultant for 
the Fire and Emergency Training Network (FETN). Linstrom’s article, “Decision Making 
on High Alert with New Threats, ” reports that most responses are to fires, emergencies, 
and medical events that are small and manageable. However, in today’s asymmetric 
threat environment, officers must be alert to responses that appear to be routine, but in 
fact are planned terrorist attacks. Lindstrom reports that 80 percent of terrorist attacks 
involve the use of explosives. 16 According to Grunnar J. Kuepper, the Chief of 
Operations with Emergency and Disaster Management Inc., Los Angeles, chemical, 
biological, and nuclear weapons pose the most significant challenges to emergency 
responders, but the weapons of choice creating an “asymmetric threat environment” are 
more often guns or explosives. 17 

At the Madrid train bombings, secondary explosive devices were placed to target 
first responders. In a Lessons Learned Infonnation Sharing (LLIS) report entitled 
Secondary Attacks: Failure to Perform Adequate Site Inspection Procedures, the Madrid 
bombings highlight concerns about secondary attacks. At this incident, all unclaimed 
personal belongings were removed to the police station. Included in these belongings was 
a bag containing an explosive device wired to a cell phone. Fortunately, the cell phone 
and explosives were not properly connected, and the device failed to detonate. Lessons 
learned underline the inadequate site security and inspection procedures, while 
underscoring the cautions that need to be taken at terrorist sites. 18 These cautions need to 
be the stimulus for enhancing first responder asymmetrical decision making on a tactical 

16 John Linstrom, "Decision Making on High Alert with New Threats, ” Fire Chief (April 2004): 29. 
[Accessed Proquest, January 2007]. 

17 Grunnar J Kuepper, “Shooting at LAX on July 4, 2004: Lessons Learned,” IAEMBulletin (October 
2002): 20. http://www.LLIS.gov. [Accessed May 25, 2007]. 

18 “Secondary Attacks: Failure to Perform Adequate Site Inspection Procedures” (September 15, 

2004). http://www.LLIS.gov. [Accessed May 25, 2007]. 
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and strategic level. By sharing lessons learned from similar incidents, first responders 
operating at subsequent incidents will be more attentive to cues and aware of their 
importance. Via experience, they will have a greater capacity to connect the past and 
present, and not miss a cue/pattern that was overlooked because the 
knowledge/experience was missing. 

Steve Lambakis et ah, report in Understanding “Asymmetric” Threats to the 
United States, that a biological weapon’s attack against a population will have a much 
different effect than the same attack in a battlefield. Although masks are an acceptable 
means of protection against such an attack, will the masks and knowledge of attack be 
available? The United States does not have the monitoring capabilities to detect all types 
of agents. Unknown or generically altered agents will not be in the monitor’s “library.” 
Additionally, the Lambakis report claims that there are millions of locations where a 
detector can be placed; obviously not every location can be protected. Attackers know 
which agents have a vaccine and will avoid the use of particular agents. Lastly, 
effective treatment depends on a rapid diagnosis. 19 

A review of the McKinsey findings reveals that first responder experience and 
prior training (emergency response drills) allowed first responders to think and act 
intuitively at the Pentagon on September 11, and to make effective decisions. Conversely, 
due to a lack of familiarity and experience concerning today’s asymmetrical threat 
environment, first responder decision making at Ground Zero 20 and Hurricane Katrina 21 
in the days following the event was not as effective. Analyses of such events have 
underscored the need for improved decision-making skills and provided tangible proof 
that recovery efforts were adversely affected when the decision-making and problem¬ 
solving tools were not available. 


19 Steven Lambakis, James Kiras, Fristin Kolet, “Understanding ‘Asymmetric’ Threats to the United 
States,” National Institute for Public Policy (September 2002): 33. http://www.nipp.org. [Accessed May 
23,2007], 

20 New York Police Department, “Improving NYPD Emergency Preparedness and Response,” (New 
York: McKinsey and Company, 2002). 
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A RAND report titled, Protecting Emergency Responders: Lessons Learned From 
Terrorist Attacks, reviewed three first responder issues in the days following a terrorist 
attack, and how prior training may have assisted decision making. The successful 
establishment of a perimeter control at the two sites (Pentagon and Ground Zero) was 
influenced by many factors; the Pentagon was in a rural setting with limited access, while 
Ground Zero was in an urban environment with many access points. However, the 
number of access points and site location were not the sole contributing factors 
determining the accomplishment of the objective. For example, in reviewing post 
operations at the Pentagon, the RAND report points out that “Federal, state, and local 
agencies involved were accustomed to working with each other and had practiced 
emergency-response drills. Several firefighters agreed: ‘What made it [effective] was the 
training that we do all the time with the other jurisdictions, commands, and knowing the 
people.’” 22 

A post-9/11 report prepared by McKinsey and Company for the NYPD identifies 
site security at Ground Zero in the days immediately following September 11, 2001, as 
inadequate. Perimeter security was not adequately established, estimating the risk of a 
second attack was not made a priority, and leadership was unclear on how to obtain 
additional resources. 23 According to Falkenrath, “While preparedness plans should be 
addressed in exercises and simulations, no exercise can substitute for real world 
experiences, and because life safety and normal life activities cannot be sacrificed, 
exercises will always contain a certain element of uncertainty until played out in a real 
life experience. The question that remains unclear is how well the training will hold up at 
a real world incident.” 24 

Communication of risk and access to vital information in an asymmetrical threat 
environment is imperative. First responders need to make decisions predicated on 

22 James T. Bartis, Irene Brahmakulam, Ari Houser, Brian A. Jackson, D. J. Peterson, Jerry Sollinger , 
Tom LaTourrette, “Protecting Emergency Responders: Lessons Learned From Terrorist Attacks.” Rand 
and Science and Technology’ Policy Institute (Arlington, Virginia, 2002): 48. 
http://www.rand.org/pubs/conf_proceedings/CF176/index.html. [Accessed May 2, 2007]. 

23 New York Police Department, “Improving NYPD Emergency Preparedness and Response.” 

24 Falkenrath, “Problem of Preparedness,” 13. 
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dynamic, changing information provided by numerous agencies — information and 
situations they may not have confronted prior to 9/11. According to Bartis et ah, a 
commander’s decision-making concerns included the following: “The desire to have 
accurate infonnation was of particular concern to commanders, who were making 
decisions about what level of risk their personnel would face. ‘You’ve got to understand 
what you’re dealing with,’ said a law-enforcement representative. ‘We can equip our first 
responders to a certain level . . . but as managers, we have to understand what risks they 
are capable of dealing with or what hazards [are out there].”’ 25 Decisions will be 
predicated on the situation and circumstances. 

For example, are we as managers/leaders prepared to commit personnel to an 
environment that has not been properly evaluated, even though life and property may be 
at risk? If no one is exiting a subway station following reports of an explosion are we 
prepared to commit personnel to that same area, knowing that they (first responders) may 
become part of the problem? If we do not make initial decisions that protect first 
responders, the first responders will not be available to assist those in trouble. 

Consider the following analogue: 

One recommendation that came out of the McKinsey and Company report 
prepared for the FDNY concerning fire department operations at ground zero was the 
expansion of procedures for exchanging operational information with other agencies. 26 In 
the concluding remarks section of the RAND document, commanders discussed the 
difficulties with managing a terrorist attack site: “Given the understandable difficulty of 
making such decisions in the midst of a response effort, site commanders could greatly 
benefit from guidelines developed in advance of an incident.” 27 


25 Bartis et al., “Protecting Emergency Responders,” 40. 

26 Fire Department of New York, “Increasing FDNY's Preparedness” /New York: McKinsey and 
Company, 2002): 57. 

27 Bartis et al., “Protecting Emergency Responders,” 80. 
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At the most fundamental level, part of the explanation for why the 
response to Katrina did not go as planned is that key decision makers at all 
levels simply were not familiar with the plans. 28 

The Federal Response Plan to Hurricane Katrina: Lessons Learned states that 
many key decision makers were not familiar with the National Response Plan (NRP), 
which led to ineffective coordination between federal, state, and city agencies. Local 
governments (first responders) were responsible for developing a plan that could be 
integrated into the NRP. Many of these plans, however, were still in the development 
stage or non-existent. The result was an operation void of SOPs, operational guidelines, 
and chain of command protocol. 29 The Federal Response Plan reports that in Louisiana, 
significant delays slowed down the delivery of pre-positioned medical, public health, and 
pharmaceutical assets: 

In some cases, security and logistics may have been issues, but delays in 
“on the ground” decision making by local and state officials resulted in 
delays in the delivery of assets and services when and where they were 
needed. 30 

2. Decision Making 

Research on the problem-solving and decision-making strategies used by today’s 
first responders indicates that many decisions requiring immediate action are often made 
intuitively, utilizing a “gut like” decision-making process. 31 Gary Klein reports that first 
responder decision-making strategies at fires and emergencies involve experience and 
intuitive “gut instinct.” 32 By definition, one would believe experience and intuition to be 
in conflict regarding decision-making styles. The fonner is built on past events tried and 
true, while the latter is a “shooting from the hip response.” In reality, however, they 


28 Federal Response Plan to Hurricane Katrina: Lessons Learned, February 2006. 
http://www.whitehouse.gov/reports/katrina-lessons-learned.pdf. [Accessed May 2, 2007]. 58. 

29 Ibid., 202. 

30 Federal Response Plan to Hurricane Katrina: Lessons Learned, 202. 

31 Klein, Sources, 33. 

32 Ibid. 
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complement each other. Experience mentally warehouses information and intuitions pull 
the knowledge from the shelves. 33 

In Sources of Power: How People Make Decisions, Klein describes the decision¬ 
making and problem-solving techniques of people who make decisions under stressful, 
time-pressured circumstances. Klein interviewed firefighters, nurses, air traffic 
controllers, and military leaders to detennine why they decide to go left or right, to run or 
stay put. What makes a fire officer decide to evacuate a perfectly sound building, only to 
have it collapse minutes later? “Firefighters rely on recognition of familiarity and 
prototypicality,” 34 Klein explains, meaning they often rely on mental models to anticipate 
what will happen next. Although one would believe that most decisions made under life- 
and-death circumstances would require a more analytical decision-making process 
reflective of the “Seven Step Problem-Solving Model” cited earlier in the chapter, Klein 
has discovered that many emergency decisions are made to the contrary. Decision makers 
rely on experience; they use their knowledge of past events to help them solve present 
day problems. Experience allows them to think intuitively, in a blink of an eye. A 
seasoned attending nurse has seen enough cases concerning an infant’s ill health to 
diagnosis a problem and institute a remedy long before a blood test can be verified. 
Nurses can do this because they are exposed to numerous trials with similar symptoms 
(patterns). When there are enough commonalities, the nurses’ perceptions and 
comprehension of the current situation are compared to their mental model of the 
situation. By connecting the past to the present, intuition assists in the decision-making 
process. 

While Klein discusses numerous strategies decision makers utilize to think and 
act intuitively, he also studies the strategies decision makers use when experience is not a 
viable tool, when decision makers are in “uncharted waters.” Using mental simulation to 
construct plausible outcomes is one such strategy. Good situation awareness facilitates 
the decision maker in diagnosing the problem; mental stimulation provides the ability to 


33 Duggan, “Coup D’oeil: Strategic Intuition in Army Planning,” 1. 

34 Ibid. 
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rehearse numerous scenarios until one works. 35 Mental simulation and the rational choice 
strategy both involve comparing options and mentally playing out the situation to see if a 
particular course of action really works. Mental simulation is a rapid decision-making 
process, while rational choice strategy is more deliberate. Also, unlike rational choice 
decision making, mental simulation employs the art of “satisficing,” namely, taking the 
first option that works, not necessarily the best. 

The recurring theme in Klein’s Sources of Power is that experience is a major 
contributor to successful decision making under pressure. However, when experience is 
not available, as is often the case in an asymmetric threat environment, experts must 
devise alternative methods to problem solve. Klein’s recognition-primed decision model 
(RPD) suggests the use of mental simulation as one alternative method when we have to 
make sense of different clues. 36 Mental simulations allow the decision maker to mentally 
rehearse a series of decisions. If they make sense, the decisions are implemented, if not, 
different actions are considered. “In order to build an effective mental simulation, we 
need to have good mental models (fonned from stored experiences) of how things 
work.” 37 

Crandall et ah, report that stored experiences assist the decision maker in 
recognizing patterns/matches during current situations, building mental models to be 
stored in memory and referenced during future situations, and recognizing similarities 
between like events. 38 The art of correctly sizing up a fire incident and deciding whether 
it should be expanded via the request for additional units, scaled down by reducing unit 
commitment, or remaining the status quo by keeping the present alann assignment, is an 
example of combining experience with critical thinking. 

Linstrom employs Klein’s recognition-primed decision model when he discusses 
the various methods a fire officer uses to size up a situation. According to Linstrom, a 

35 Klein, Sources, 89. 

36 Ibid. 

37 Gary A. Klein, The Power of Intuition (New York: Currency Doubleday, 2004): 27. 

38 Beth Crandall et al., “Training Decision Makers for the Real World,” Decision Making In Action: 
Models And Methods (ed). Roberta Calderwood et al. (Norwood, New Jersey: Ablex Publishing 
Corporation, 1993) 312. 
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first responder matches cues/clues from the current event with stored experiences from 
past events. These experiences are stored as templates or schemas. The decision maker 
compares cues such as the color of smoke (black, gray, white) with the cues experienced 
in previous fires to help assess what kind of fire or stage of fire they are dealing with. 
Once a pattern is matched, the response action that worked in the previously experienced 
situation will be recalled and can be applied. By making this comparison, a first 
responder can mentally draw from similar situations, recalling which decisions were 
effective, and decide on an appropriate course of action for the current situation. From 
this analysis, a course of action is developed. It may not be the best or only choice, but it 
is one that works. 39 


Imagine that a fire officer is responding to an alarm reporting an explosion inside 
a building. Unbeknownst to the officer, this is a terrorist-related incident. According to 
Linstrom, without prior knowledge of these facts and not possessing the 
experience/training to size up an incident and detect possible terrorist related activities, 
the officer might commit his units to an interior attack, exposing them to a possible 
secondary device. 40 


The fire officer may find himself at a distinct disadvantage because he does not 
have the experience to dissect the situation — nor the SA that would help him create 
pattems/matches with previously-experienced situations to develop similarities. This is 
because terrorism is an asymmetric event, and many fire officers have not yet developed 
the expertise to think and act decisively in these novel situations. Instead, they will rely 
on a more traditional, rational decision-making model. Linstrom calls this a “no slide in 
the slide tray” moment: The cues are foreign to the officers’ experiences and training. 41 


39 Linstrom, “Decision Making,” 30. 

40 Ibid. 

41 Ibid„ 32. 
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“Slide tray” slides are synonymous for the long-term memory stores (also referred to as 
“schemas” or templates) that often separate a novice from an expert. 

3. Situational Awareness 

Simply stated, situational awareness (SA) is the art of understanding your 
surroundings while developing three distinct levels of environmental awareness, namely 
perception (level 1), comprehension (level 2), and projection (level 3). 42 

For example, an experienced fire ground commander, moving through the SA 
levels of perception, comprehension, and projection, quickly sizes up the scene 
(perception), understands the possibilities and ramifications of fire spread 
(comprehension), and quickly orders a response that complements the present and future 
status of the fire (projection). 

Effective first-responder decision makers create parallels between SA levels of 
perception, comprehension, and projection and RPD pattem-matching/cue 
recognition/mental model construct. Is SA part of the decision-making process? In 
Theoretical Underpinnings of Situational Awareness: A Critical Review, Endsley reports 
it is not. Endsley feels that SA is the precursor to decision making. SA provides a 
representation of the environment; then the decision maker decides what to do, based on 
the information. 43 According to Endsley, effective decision making, leading to effective 
performance, often depends on high levels of SA. 44 But, good SA is not a guarantee for 
good decisions. For example, SA could be correct, yet the wrong decision is made. 

Marvin Cohen tells us that good SA helps experts to solve problems in familiar 
situations by pattern matching similarities. By first developing an accurate SA helps 
decision makers make sense of unfamiliar surroundings. Cohen states that one way to 


42 Mica R. Endsley, “Expertise and Situation Awareness,” (eds) Neil Charness, K. Anders Ericsson, 
Paul J. Feltovich, and Robert R. Hoffman, Expertise and Expert Performance (New York: Cambridge 
University Press, 2006): 634. 

43 Endsley, “Theoretical Underpinnings of Situational Awareness,” 4. 

44 Endsley, “Expertise and Situation Awareness,” 634. 
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attain this skill is to have decision makers think more critically about novel situations. 45 
Klein, in his recognition-primed decision-making model, sees a direct linkage between 
recognition of the situation and actions taken. 46 Cohen agrees that recognition is 
imperative to the process; however, there are other factors to be considered. For example, 
what happens if the SA is inaccurate? Cohen suggests the process of meta-recognition to 
verify and improve recognition results. Meta-recognition probes for mistakes, corrects 
what it finds, and evaluates the results. Cohen believes that by studying the strategies that 
experienced decision makers use when dealing with uncertainty, training can be 
enhanced. 47 This is encouraging because it shares Klein’s view of cognitive modeling 
and expert/novice contrasts. 

4. Expert vs. Novice: Characteristics of Expert Decision Making 

When confronting a naturalistic decision-making (NDM) situation characterized 
by uncertainty, dynamic conditions, ill-structured problems, and time constraints, experts 
focus on comprehending the situation first and developing a course of action in the 
decision-making process second. 48 Experts will spend more time developing situational 
awareness and, once they recognize the problem as familiar, they implement a course of 
action. Conversely, novices spend more time comparing different options because they 
do not have the stored experiences (mental models) that would help them recognize the 
problem. 49 

Richard Adams of the Florida Institute of Technology advises the Federal 
Aviation Administration (FAA) on aeronautical decision making. Of particular 
importance to Adams and the FAA are the decision-making skills of pilots and how they 

45 Marvin S. Cohen and Bryan B. Thompson, “Training Teams to take Initiative: Critical Thinking In 
Novel Situations,” Cognitive Technologies, Inc (September 11, 1999): 5. http://www.cog- 
tech.com/Publications/PubsRM.htm. [Accessed April 10, 2007]. 

46 Ibid., 4. 

47 Marvin S. Cohen and Jared T. Freeman, “Thinking Naturally About Uncertainty,” Cognitive 
Technologies, Inc (n.d.):l. http://www.cog-tech.com/papwer/humfac/hf_1996.pdf [Accessed April 2007]. 

48 H. Lauren Pugh, Josephine M. Randal, and Stephen K. Reed, “Differences in Expert and Novice 
Situation Awareness in Naturalistic Decision Making,” International Journal of Human-Computer Studies 
45, no. 5 (November 1996). 580. [Accessed Pro Quest August 2007]. 

49 Ibid., 582. 
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overcome adversity in everyday flights. Adams’ identification of seven distinguishable 
characteristics of expert decision making is gennane to many fields of expertise (e.g., 
pilots, first responders, nurses): 50 

1. Experience and training develop an expert’s knowledge base. This 
knowledge base allows the expert to perceive large meaningful 
patterns. The experts see in “chu nk s” rather than individual snippets of 
information. Because pattern recognition occurs so rapidly, it is 
sometimes described as being intuitive. 

2. Because an expert can construct associations between the past and 
present, short-term working memory is available to address the 
problem at hand and the expert is not pre-occupied attempting to make 
sense of existing circumstances. 

3. Expert knowledge is procedural and goal oriented. Experts know SOPs 
and understand situations where procedures are useful. 

4. Experts solve problems faster because experts are (1) faster at skill- 
based tasks; (2) more working memory capacity is available; and (3) 
employ intuitive decision-making strategies (arrive at solutions faster 
without extensive search). 

5. Experts excel at both routine and adaptive decision-making 
opportunities. They can lean on knowledge or improvisation when 
confronting everyday or novel decision-making situations 

6. Experts have the capability to revise procedures while validating 
situational interpretations through the process of mental simulation. 

7. Because experts possess greater domain-specific knowledge, they 
spend more time analyzing and predicting problem complexity 
predicated on the underlying principles of a problem. This reflection 
results in better management regarding the allocation of time to 
problem solving. 51 


Good situational awareness and comprehensive mental models are two reasons 
experts are better decision makers. First, when advancing through the various levels of 


50 Dick McKinney, American Airlines, Retired. Decatur, Texas, “On Improving Tactical Decision 
Making.” 3. http://www.crm- 

devel.org/resources/paper/On%20Improving%20Tactical%20Decision%20Making%20McKinneD.pdf . 
[Accessed November 22, 2007]. 

51 Ibid., 3. 
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SA, experts expend more time making sense out of uncertain environments. While a 
novice will become overwhelmed in the complexities of situational comprehension (SA - 
Level 2), the expert will be selective regarding what is and is not relevant to the current 
situation. This selectivity translates into limited choice selection regarding SA -Level 3 
projection. The expert has less to consider because they are able to eliminate non- 
essential cues, they are therefore more likely to make a suitable choice that solves the 
problem at hand. Conversely, the novice in the same situation finds himself or herself 
tangled in a decision-making quagmire attempting to select the best course of action from 
a diverse menu of self-created options. Second, exposure to numerous decision-making 
situations has allowed the expert to develop effective mental models. As experience 
grows, the expert’s mental models became more complex and comprehensive. 

According to Crandall et ah, an expert knows how to use his/her domain 
knowledge, while the novice relies on general knowledge. 52 Experts have numerous 
advantages; they see incidents in tenns of patterns, they plan and they do not need to 
consider numerous alternatives. 53 An expert takes the first option that works; a novice will 
consider many options. Experts frame the problem looking for similarities and pattern 
matches; a novice will allow the complexities of an incident to confuse what is relevant 
and what can be discarded. The good news is that as experience and knowledge grow, 
novices often turn into experts. 54 


Table 1. Comparison of Expert and Novice Decision-Making Characteristics 


Expert 

Novice 

Spends more time developing SA 

Spends more time weighing options 55 

Uses domain knowledge 

Relies on general knowledge 

Frames problems by finding similarities 

Allows complexities of incident to confuse 
decision-making 

Sees in patterns and does not consider multiple 
options, takes first option that works 

Considers multiple options 56 


52 Crandall et al., “Training Decision Makers for the Real World,” 312. 

53 Ibid. 

54 Ibid., 311. 

55 Pugh, “Differences in Expert and Novice Situation Awareness in Naturalistic Decision Making,” 

580. 

56 Crandall et al., “Training Decision Makers for Real World,” 312. 
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Comparing the proficiency of an expert and novice in their ability to develop SA, 
an analogy can be made to driving on the freeway. In a paper titled “Anticipatory 
Thinking,” Klein et ah, discuss the anticipatory thinking that often overlaps with the 
prediction that occurs in Level 3 of SA. For example, while an experienced driver will 
scan the road for potential problems, using their peripheral vision to take in the “big 
picture,” an inexperienced driver will be more focused, concentrating not on potential 
problems, but rather on road issues in their immediate line of sight — namely, keeping 
their car between the white lines while ignoring potential trouble spots. 57 The novice’s 
SA is handicapped via a “tunnel vision” mentality regarding environmental awareness. 
They are unable to process auxiliary infonnation because all attention is focused on the 
basic task itself. Because the experienced driver can “automatic pilot” certain aspects of 
their driving SA and decision making, they are more attentive to the weak signals that 
may indicate a future problem, thereby recognizing a small problem while it is small and 
manageable. 

To presume that experts do not think before they act is not completely accurate. 
While their thought process may be ongoing and non-reflective, experts do, time 
pennitting, ponder their options. Experts, however, do not allow themselves to get caught 
up in the rules and reasons behind a decision, because if the did they would be reverting 
back to a novice decision maker. 58 Rather they reflect on the goals and actions to achieve 
them. This process is often referred to as “deliberate rationality.” However, this 
cognitive process should not be confused with a rational choice analysis, but rather one 
that understands goals and objectives and how best to achieve them. 59 Experts employ 
deliberate rationality with the intent to improve one’s intuitive behavior without resorting 
to the more theory-based actions a novice may rely on. 60 The expert has synthesized SA 

57 Gary Klein, Chew Lock Pin, and David Snowden, “Anticipatory Thinking,” (eds) U. Fisher and K. 
Mosier, Proceedings of the Eight International NDM Conference (Pacific Grove, California, June 
2007): 120. 

58 Hubert L. Dreyfus. “Intuitive, Deliberate, and Calculative Models of Expert Performance,” (eds) 
Gary Klein and Caroline E. Zsambok, Naturalistic Decision Making, Lawrence Erlbaum Associates, 
Mahwah, New Jersey, 1997): 28. 

59 Ibid., 28. 

60 Ibid. 
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levels 1 and 2 by considering goals and objectives (perception and comprehension) and, 
when time permits, using available information to make a level 3 SA (projection) 
decision. 

In Dreyfus’ chapter, “Intuitive, Deliberative, and Calculative Models of 
Performance,” he discusses philosopher Martin Heidegger’s three kinds of skilled 
responses to a situation. Heidegger says there are three methods of coping, namely ready- 
to-hand, present-at-hand, and unready-to-hand. (1) Everyday coping can be described as 
ready-to-hand; we react without any thought, (2) present-at-hand coping involves those 
circumstances that are unfamiliar and where experience is irrelevant, and (3) when we 
resort to unready-to-hand we usually deal with a situation where the decision maker has a 
great deal of domain experience. They do not react intuitively, however, and instead stop 
and think. This coping should not be confused with rational deliberation. Unlike dealing 
with the unknown, deliberation takes place in a familiar context. Reasons for this coping 
process can include situations that are unusual or those requiring justification. 61 

Heidegger’s analysis is interesting for several reasons. First, ready-to-hand 
coping is a relative term because decisions that seem easy to some may be extremely 
difficult to others, merely because of domain knowledge. Ready-to-hand is a good 
example of previously discussed first responder experts. Second, present-at-hand coping 
provides no resident expert. For example, “space exploration” has little available 
experiential information so has little value to research. Fastly, first responders operating 
in an asymmetrical environment (novice) appear to be closely identified with unready-to- 
hand decision making. They have some experience, but matters at hand are somehow 
unusual. 

A question concerning the expert/novice relationship may not be how we get a 
novice to think like an expert, but instead, how we get a novice to learn like an expert! 
An expert learns by engaging in deliberate practice, compiling an extensive experience 


61 Dreyfus, “intuitive, Deliberate, and Calculative Models,” 27 
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bank, obtaining feedback, and gaining new insight and lessons from mistakes. 62 Crandell 
et al., describe how experts fine-tune their knowledge by: 

chunking information, recognizing patterns and being aware of the critical 
information while disregarding less important information...decisions in a 
natural setting are determined by individual experiences, pattern 
recognition and creating an association between familiar patterns and 
actions...through years of experience and thousands of trials, experts come 
to see the world in patterns. 63 

In Table 2, Klein identifies skills that can be taught, along with methods designed 
to make individuals better decision makers. Klein’s statement that these skills are not 
generic suggests that trainers must tailor the training to meet decision maker’s specific 
needs. 64 


Table 2. Skills to be Trained and Methods 


Skills Trained 

Methods for Training 

Situational awareness, pattern matching, 
cue learning 

Strategies for considering alternative 
hypothesis / explanations 

Typical cases/anomalies 

Strategies to detect earlier warning signs 
/contingency preparation 

Mental models/mental simulations 

Cognitive modeling and expert /novice 
contrasts 

Managing uncertainty/time pressure 65 

Tactical decision games / develop 
metacognitive skills 


5. Rational Choice and Naturalistic Decision Making 

An important prerequisite for effective rational choice problem solving/decision 
making is the availability of vital information; time to compare options, well-defined 
goals, and static conditions. Rational choice decision making is reliable, it helps novices, 
it is rigorous, and, in general, can be applied to other situations. 66 The characteristics and 


62 Klein, Sources, 104. 

63 Crandell et al., “Training Decision Makers for the Real World,” 314. 

64 Gary Klein, “An Overview of Naturalistic Decision Making Application,” Naturalistic Decision 
Making, (eds) Gary Klein and Caroline E. Zsambok (Mahwah, New Jersey: Lawrence Erlbaum Associates, 
1997), 53. 

65 Ibid., 54. 

66 Klein, Sources, 29. 
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urgency of a decision-making event often determines the feasibility and practicality of a 
particular decision-making strategy. This is not to suggest that individuals ponder 
decision-making strategies before addressing a problem. Quite the contrary, decision 
makers make decisions as best they can, unaware of how they do it. The strategies they 
use are a product of experience, rather than an intimate discussion with oneself, arguing 
the advantages of rational choice over intuition! In Sources of Power, Klein identifies 
situations where a rational choice strategy may be the best choice: 

• Is there a need for justification; will actions be scrutinized by a higher 
authority? 

• Conflict Resolution: need to place all considerations into a common 
format 

• Optimization: finding the best course of action, time to compare (time 
is not a factor) 

• Lack of experience: lack of portfolio of experiences 

• Conditions are static 

• Goals are defined 67 



Figure 2. Conditions Favoring Intuitive and Analytical Approaches. 68 


67 Klein, Sources, 96. 
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Thomas Karp, a Ph.D. candidate at Rushmore University, stated, ‘'Intuition and 
analysis (rational choice) are essential complementary components of effective decision 
making,” 69 which suggests a mutual dependency between the two strategies. In Strategic 
Intuition in Army Planning, William Duggan, an associate professor at Columbia 
Business School, explores the idea of combining the decision-making strategies of 
intuition and rational choice to form a more complete strategy. 

Duggan addresses the concept of “intelligent memory,” which is a union of 
intuition and rational choice decision making. He makes an interesting analogy between 
the human mind and a warehouse: we stock our warehouse shelves (human brain) with 
information obtained from knowledge and experience, storing what we view as 
important; this is analysis (rational). According to Duggan, when confronting a decision¬ 
making opportunity, intuition utilizes the stock from our shelves: 70 “The divide between 
analysis (rational) and intuition reflects an outmoded view of the human mind that 
science no longer supports.. .In the new view, analysis, and intuition are so intertwined 
that it is impossible to separate the two. Some scientists call this ‘intelligent memory,’ 
where analysis puts elements into your brain and intuition pulls them out and combines 
them into action.” 71 

In Thinking Naturally About Uncertainty , Marvin Cohen asks the question, “Has 
natural decision making matured enough to face uncertainty head on?” 72 Cohen proposes 
several “what if’: scenarios. For example, when patterns and similarities do not exist, 
can the decision maker use mental simulations to rehearse possible outcomes? How do 
you mentally rehearse a course of action when you have no prior operational experience? 
Possibly, using story building to help organize events while relating a real-life experience 
regarding a similar event to the problem at hand may be one option. 

68 Gary Klein, The Power of Intuition (New York: Currency Doubleday, 2004): 67. 

69 Thomas Karp, “Intuitive Decision Making in Strategy Management,” Research Paper-Rushmore 
University (2002): 11. http://tom.karp.as/idm.pdf [Accessed January 3, 2007]. 

70 Columbia Ideas at Work, “Strategic Intuition: The Keys to Innovation,” Columbia Business School 
(Summer 2006): http://www.gsb.columbia.edu/ideas. [Accessed January 2006]. 

71 Duggan, “Coup D’oeil: Strategic Intuition in Army Planning,” V. 

72 Ibid., 2. 
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6. Computer-Based Simulation Technologies 


The military conducts war games designed to train commanders, soldiers, naval 
battle groups, and wings of an aircraft. In his article, The Application of Existing 
Simulation Systems To Emergency Homeland Security Training Needs, Smith suggests 
modifying war games by replacing the military with the fire service. Smith says that a fire 
chief could replace a military commander, firefighters replace soldiers, and fire trucks 
replace naval battle groups. The new enemy would be a fire, chemical release, or a 
building collapse. Fire ground commanders would be required to solve problems and 
make decisions based on the map display interface of a building instead of a battlefield. 73 
Today’s technology makes this idea a possibility while providing first responder training 
with a “tool” that has already been field-tested. First responder communities, much like a 
military unit, are often required to problem solve and make choices under stressful 
conditions. An asymmetrical threat scenario designed to evaluate current decision¬ 
making capabilities can be developed for this simulation system. 

Douglas Page is a science and technology writer. In his Fire Chief article, 
“Simulator Goes to Head of Class,” Page asks the question, “How do you prepare for 
your first blaze/fire situation that you have never been in before?” 74 Page reports that 
current virtual reality fire simulators are being retrofitted to demonstrate how life- 
threatening fire conditions can develop. In his article, “Make the right call,” Charles 
Burkell, the program chair for executive development at the National Fire Academy, 
suggests the following concerning simulation training: 

Fire service organizations need to include time pressure, shifting 
conditions and information gaps in training exercises. Simulations and 
vicarious experiences are a way to build proficiency in lieu of actually 
having “been there and done it.” 75 


73 Roger Smith, “The Application of Existing Simulation Systems To Emergency Homeland Security 
Training Needs,” Titan Corporation, Simulation Interoperability Workshop (Europe 2003): 3. 
http://www.modelbenders.com/papers/03E-SIW-012.pdf. [Accessed June 15, 2007]. 

74 Douglas Page, “Simulator Goes to Head of Class,” Fire Chief (December 2003): 14. [Accessed 
Proquest, January 2007]. 

75 Charles J. Burkell, “Make the right call,” Fire Chief (March 1999): 45. [Accessed Proquest, January 
24, 2007]. 
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Refinements to training include tactics tested on a computer without risking life and limb, 
simulated smoke, hot air, fire gases, and wind ventilation ramifications. 

According to Page, from a decision-making perspective, first responders will be 
able to observe the implications of their actions. 76 The ability to immediately observe 
implications of actions is a major advantage of simulator training, according to Klein. 77 
Burkell adds that 

There are no foolproof strategies. A fundamental principle in the military 
is that commanders must make decisions within times of uncertainty, 
under great pressure. It is just part of the business. While highly 
experienced command officers strive for “zero defects,” the reality is that 
some level of error will exist. Improving one's ability to make the right 
decisions relies on supplementing actual experience with simulation-based 
training. 78 

Burkell believes there are not enough real-world opportunities for first responders 
to develop effective decision-making skills, especially when operating in a world of 
uncertainty. However, simulation training - creating computer-based scenarios and 
situations that compel first responders to make choices - is a conduit to developing 
decision-making skills. 79 

In her article, “Collective Essentials for Test Pilot Leadership Training,” Roxana 
Tiron, a writer for National Defense, reports that to test the decision-making and 
communication skills of military pilots, instructors will often introduce ad hoc injects or 
“curveballs” into the scenario. 80 These last-minute inserts are designed to disturb the 
pattern matching/recognition process that the intuitive decision maker relies on. 


76 Page, “Simulator Goes to Head of Class,” 14. 

77 Crandall et al., “Training Decision Makers for the Real World.” 314. 

78 Burkell, “Make the right call,” 43. 

79 Ibid., 43. 

80 Roxana Tiron, “Collective Simulation Essential For Pilot Leadership Training,” National Defense 
(December 2004): http://www.nationaldefensemagazine.org/issues/2004/Dec/CollectiveSimulation.htm 
[Accessed June 15, 2007]. 
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According to Tiron, when pilots make a bad decision during simulation training, they 
have the opportunity to go back and determine where they went wrong. 81 

7. Decision Skills Training 

“Decision skills training (DST) addresses domain-specific training designed to 

accelerate the transition toward expertise.Rather than attempting to teach decision¬ 
making strategies, DST attempts to enhance a decision makers experience bank (mental 

models) in specific domains, thereby increasing their recognition decision-making skills. ^3 
DST is based on a list of training strategies that provide opportunities for practitioners to 
learn more rapidity. Namely: (1) engaging in deliberate practice; (2) obtaining feedback that 

is accurate; (3) building mental models; (4) recognizing opportunities to learn.84 

DST is often carried out via a low-fidelity simulation exercise (pencil and paper) 
referred to as tactical decision games (TDG). Using a facilitator-led discussion, 
participants provide responses (decisions) predicated on carefully constructed scenarios 
that are domain-specific. For example, scenarios are read aloud to specific decision 
points, and participants are then given an agreed upon time limit to prioritize decision 
making predicated on an existing scenario. According to Klein et al., “Tactical decision 
games add to the trainees' experience base, prepares them to respond under uncertainty 
and time pressure, and requires them to formulate their intent.” 85 Real world dilemmas 
often include high levels of uncertainty and time pressure. Participants are given a finite 
time limit regarding game-decision responses. 86 


81 Tiron, “Collective Simulation Essential For Pilot Leadership Training.” 

8 - Gary Klein, Karol G. Ross, Jennifer L. Schafer, “Professional Judgments and Naturalistic Decision 
Making,” (eds) Neil Charness et al.. The Cambridge Handbook of Expertise and Expert Performance (New 
York: Cambridge University Press: 2006): 412. 

85 Gary Klein, Rebecca M. Pliske, Michael J. McCloskey, “Decision Skills Training: Facilitating 
Learning From Experience.” (eds) Gary Klein and Eduardo Salas, Linking Expertise and Naturalistic 
Decision Making (Mahwah, New Jersey: Lawrence Erlbaum Associates, 2001): 40. 

84 Ibid., 41. 

85 Ibid. 

86 Appendix C. Decision Skills Training, C-l, http://www.nifc.gov/safety/phaseiii_chapters/WFSAS- 
AppendixC.pdf. [Accessed December 10, 2007]. 
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DST scenarios are designed so novices and experts alike experience the 
following: (1) limits of their current repertoire of mental models; (2) practice in cue 
recognition/pattem matching, goal identification and checking expectancies; (3) 
practicing mental simulation via mental models; (4) receipt of expert feedback regarding 
mental models and COAs; and (5) comparing their decisions with others. Because expert 
knowledge is believed to be tacit knowledge, it is difficult to share; how can you tell 
someone to perform an unconscious process? There are no lectures on how to make 
decisions. Instead, a realistic scenario incorporating recognizable cues and patterns 
provide novices the opportunity to perform and reflect upon their decisions. Scenario 
problems require both SA and a decision-making process. Upon completion of decisions, 
a facilitator-lead discussion provides trainees with expert feedback and a structured 
dialogue on recognition and mental simulation process. 87 

Incorporating the use of tactical decision games, or a similar method into first 
responder decision-making training should receive careful consideration for the following 
reasons: 

• They provide a good opportunity to practice the skills that would be required in 
the management of an emergency situation. 

• Participants make decisions while considering the consequences of a selected 
course of action, and have the opportunity to compare this with other possible 
courses of action. 

• Decision makers evaluate the rationale behind decisions, rather than only focusing 
on the decision made. 

• No limits are placed on the decisions that can be made. They are not script-driven. 

• Scenario-based exercises allow participants to explore alternative task strategies, 
to collect an extensive experience bank, and to enrich experiences. 

• Participants develop a repertoire of patterns of response; opportunities to practice 
recognition-primed, rule-based and knowledge-based decision making. 

• Trainees receive immediate feedback from peers about their solutions to the 
scenarios. 88 


87 Klein, “Professional Judgments and Naturalistic Decision Making,” 413. 

88 Margaret T. Crichton, Rhona Flin, William A. R. Rattray, “Training Decision Makers - Tactical 
Decision Games,” Journal of Contingencies and Crisis Management 8, no. 4, December 2000: 211. 
[Accessed JSTOR December 2007]. 


30 



8 . 


Literature Summary 


References in the literature from real-world terrorism and natural disasters events 
highlight the need for greater first responder operational awareness for this new 
environment. Experts in the field of counterterrorism (e.g., Falkenrath and Hoffman) 
believe we are entering a period of first responder asymmetric warfare, with the growing 
threat of chemical, biological, radiological, nuclear and explosive (CBRNE) incidents to 
be a real possibility. Klein et ah, describe the decision-making strategies that individuals 
may use when required to problem solve under stressful conditions, often comparing a 
novice to an expert. Duggan takes the process a step further by proposing a possible 
unification of strategies to create more effective decision makers. Additional research 
needs to identify the differences in decision-making strategies between conventional 
(routine) and unconventional (asymmetric) environments and how to bridge this gap. 
Training provides methods for enhancing first responder decision-making expertise by 
including computer-based simulations and scenario-based instruction. 

D. HYPOTHESIS 

Through experience and training, today’s first responders have acquired the 
decision-making skills to think and act decisively when operating at fires, emergencies 
and other related events. They demonstrate these skills at the numerous real-world 
incidents they routinely respond to and mitigate. However, first responders may lack 
experience for responding to asymmetrical scenarios, and lack the expertise to 
successfully think through and mitigate incidents that are often novel in nature and filled 
with uncertainty. 

Several strategies, both on the national and local level, have addressed this new 

threat. For example, first responder protocol is required to become compliant with the 

National Incident Management System (NIMS). Included in this initiative is the 

application of an Incident Command System (ICS) designed to assist an Incident 

Commander when managing significant events. At a local level, many cities are writing 

Emergency Response Plans (ERP) for CBRNE responses. The FDNY recently published 

an ERP for a radiological response. While these initiatives are forward thinking, they still 
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lack one key ingredient, which is experience, and experience entails exposure to variants 
of the expected situation with opportunities for trial-and-error learning in order to master 
the type of responses that will be required. 

In essence, this asymmetrical threat environment places the expert decision maker 
in situations where they think and react more like a novice due to the lack of an 
experience base from which to draw. The goal of this thesis is to develop a strategy for 
developing decision-making expertise for this asymmetrical environment. This strategy 
will be developed by identifying the problem (lack of experience) along with 
implementing methods that have been successful in the past such as simulation and 
scenario-based training. Having participants develop a course of action for responding to 
these asymmetric threats is anticipated to fast-forward the acquisition of experience. 

The literature reveals that first responders will not be afforded the luxury of time 
when confronted with these new asymmetric threats. They need to develop alternative 
ways to acquire the decision-making skills they are unable to acquire through experience. 
Decision-making strategies of pattern matching, cue recognition, and mental modeling 
assist fire ground commanders when making “gut-like” intuitive decisions. Can these 
same skills be exploited and applied to an asymmetrical circumstance such as a 
radiological attack? I believe they can. 

As previously discussed, experience allows us to: (1) match a pattern stored in 
memory based on a preceding experience with the situation the decision maker is 
currently experiencing, (2) see similarities between present and past events in terms of 
actions that will be successful, and (3) make rationalizations concerning the current 
situation and use cues to help problem solve. A first responder’s ability to develop 
accurate SA when confronting a challenging problem can be enhanced in numerous 
ways. One way is to create deliberate practice, making sure they establish goals, have 
practice in making the types of difficult judgments that the job requires, and rehearse 
decision skills that need improvement. Another method is to expose the first responder to 
numerous decision-making situations via simulator/scenario-based training; the trainer 
can provide direct feedback and evaluation, creating a correlation between SA and 
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decisions made, good or bad. 89 Skills requiring remedial training can then be separated 
from the practice and exercised separately. Developing strategies for considering 
alternative explanations about what is really going on — not becoming fixated with your 
SA and being flexible in your initial judgment — is another method for enhancing 
decision-making skills. 

Today’s emergency responders are making decisions in a world filled with 
uncertainties, employing decision strategies that rely on a combination of a naturalistic 
(intuitive) style that relies more on experience and “gut instinct” when making choices 
and a traditional (rational) decision-making process that is methodical, regimental, time 
consuming, and deliberate in its choice selection (decision-making steps). Developing 
good decision-making skills will enable a decision maker to think critically while 
remaining flexible when managing uncertain, asymmetric events. Consider the following: 


You are a fire lieutenant working a night tour in Ladder Co. 198. You are new to 
the area and unfamiliar with the types of buildings in your response district. At 0330 
hours, you receive a water flow alarm for a local factory. Upon arrival, you observe no 
indications of fire and the building is well sealed. The dispatcher tells you there is no 
additional information on the alarm. A few of the firefighters inform you that they get this 
alarm occasionally. There is a small discharge of water coming from the side of the 
building. The chief is not available for the alarm. It is getting cold! 


Regarding the lieutenant’s actions/options at the reported water leak response, 
was there not room for both a rational and intuitive decision-making thought process? If 
the lieutenant mentally rehearsed possible reasons for the water flow alann and then 
performed a methodical investigation regarding the source of the alarm, was he not 
making decisions by using characteristics of the situations (namely, static conditions, 
need for justification, time to compare, uncertainty)? 


89 Klein, Sources, 104. 
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Is decision-making training designed to turn a novice into an expert decision 
maker a guarantee of effective decisions? Absolutely not! Numerous factors such as 
motivation, ambition, and level of effort must be considered. However, if a novice 
decision maker learns to act like an expert when operating in an emergency environment, 
would it not make sense to assume that expert decision makers can learn the same skills 
to make decisions and solve problems when operating at natural disasters and terrorist 
events? Experienced decision makers have the “skills” and “tools” already at their 
disposal. The question is how to retrofit these decision-making skills to include an 
uncertain, asymmetrical threat environment. Cohen addresses this issue when he asks, “Is 
there a naturalistic way to handle uncertainty ?” 90 By providing exposure to the skilled 
performance that reflect experts’ intuitive thought processes during everyday 
emergencies, today’s less experienced decision makers can begin acquiring the same 
skills through structured simulation/scenario-based training, and receiving feedback on 
their perfonnance. The expert/novice contrast is designed to persuade novices to learn 
from experts, thereby enhancing their decision-making process at asymmetrical threats 
that are plagued with uncertainty, surprise, and chaos. 

E METHODOLOGY 

In addition to the literature review, an in-depth analysis of first responder 
situational awareness, naturalistic decision making, and decision-making stratagems were 
conducted. Where applicable, real-world examples are used to create a connection 
between theory and reality. Finally, a scenario-based experiment was conducted. 
Experimental and control groups’ responses were compared against each other and with 
an expert group to detennine the influence/effectiveness of training and expert feedback 
during a time-sensitive exercise. Results from the experiment were analyzed and 
recommendations made. 


90 Klein, Sources, 4. 
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F. 


SUMMARY 


The remaining thesis chapters will discuss, in greater depth, the decision-making 
theories/strategies that are part of the first responders decision-making matrix, while 
striving to unite decision-making theory and reality. Real-life scenarios provide 
applicable examples designed to illustrate the relevance of first responder decision¬ 
making training in today’s asymmetrical threat environment. Chapter II will discuss the 
three levels of Situational Awareness (SA) along with supporting cognitive mechanisms; 
Chapter III defines naturalistic decision making including cue recognition, pattern 
matching, mental models, and mental simulations; Chapter IV will orchestrate and 
articulate a stratagem to transform first responders into better decision makers when 
confronting uncertainty, while identifying the characteristics that constitute an effective 
decision-making strategy. Throughout the course these chapters, scenario-based examples 
will connect theory with reality. 

Chapter V expands upon the aforementioned scenario-based learning while 
clarifying the experimental design facet of this research. This scenario-based experiment 
tasked two groups of first responders (experts and novices) to problem solve and make 
decisions regarding several realistic asymmetrical-type scenarios. The group of novices 
(experimental) received supplementary information/training (independent variable) while 
the other group (control) received none. 
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II. SITUATIONAL AWARENESS 


Chapter II identifies and defines the three levels of situational awareness (SA) 
known as (1) perception (observation of the incident), (2) comprehension (understanding 
what you observe), and (3) projection (taking action predicated on understanding). 
Additionally, Chapter II explains the supporting cognitive mechanisms that are part and 
parcel of the development of SA while describing their influence on an effective 
decision-making/problem-solving process. 

“SA is defined as the perception of the elements in the environment within a 
volume of time and space, the comprehension of their meaning and the projection of their 
status in the near future.” 91 Individuals perceive an incident, comprehend their perception 
(understand the relationships between perceived cues and the current surrounding 
environment and desired goals), and — through their analysis of the correlation between 
perception, comprehension, and desired goals — project future events. 


Imagine a seasoned fire officer and a newly promoted probationary firefighter 
standing in front of a burning building, trying to size up the situation. The building in 
question is a four-story wood frame building in a row of similar attached buildings. The 
fire is on the top floor. Both individuals perceive the same set of circumstances (top floor 
fire). However, while the seasoned officer understands that the fire will now begin to 
spread horizontally, the less experienced firefighter may not. Using existing cues, both 
individuals have entered level 2 SA. One of them, however, is at a distinct disadvantage! 


Experienced fire commanders begin their decision-making process by perceiving 
the elements of an event (fire conditions and resources present) and then comparing these 
observations with templates stored in memory. These templates are based on previous 
training and experience. For example, do operational goals include extinguishing the fire, 


91 Endsley, “Theoretical Underpinnings of Situational Awareness,” 3. 
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or rescuing occupants and protecting exposures while conceding the fire building? 
Integrating multiple pieces of infonnation and comprehending the relationship between 
elements of the problem situation and operational goals is significant in the decision¬ 
making progression. 

Continuing with the earlier scenario, the fire commander conducts an exterior 
evaluation (size-up) of a top floor fire in a row of attached wood-frame buildings 
(perception). Understanding that fires have a tendency to extend horizontally when they 
can no longer extend vertically (comprehension), the commander orders the stretching of 
a hose line to the top floor of the adjoining attached structure to stop possible extension 
of fire (projection). However, by the time the hose line reaches its intended destination, 
the fire has already gained a foothold in the adjoining structure. When the desired 
objectives/goals of fire extinguishment (prevent fire extension) are in conflict with 
existing conditions (fire in adjoining exposure), the commander makes contingency plans 
by calling for additional help (projection). 

While a large majority of SA development involves the systematic process of 
collecting, understanding, and forecasting, the transition from one SA level to the next 
does not always follow a linear pattern. For example, while progression from one level to 
the next may be indicative of a novice decision maker confronting uncertainty or novel 
circumstances, an expert with good schemata may not need to process information 
through levels 2 and 3 SA. 92 Perception or size-up (level 1) leads to projection or action 
(level 3). Good schemata results in minimal conscious deliberation. Comprehension is 
less deliberate and more intuitive. In brief, a schema is a person’s prototypical mental 
model for certain situations; they recognize the situation as typical and have a 
predetermined “game plan” or schema to implement. 93 Mental models, stored in long¬ 
term memory, help to explain the way the world works. “Mental models provide the basis 
for describing causal relations among events, for predicting what will happen as a 


92 Endsley, “Expertise and Situation Awareness,” 639. 

93 Ibid., 639. 
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consequence of an event...” 94 Prototypical situations can be learned through experience 
or deliberate practice. 95 For example, deliberate practice in the form of 
simulations/scenario-based exercises can be designed around patterns of recognition that 
would normally take years to experience naturally. 96 Because the infonnation is stored in 
long-term memory, the expert recalls the schema and applies it to the current situation. 97 

Use of schemas (mental models) is one type of decision-making strategy for 
managing a problem. Therefore, it would seem logical to believe that the greater the 
number of opportunities a first responder has to utilize stored schemas in the decision¬ 
making process, the faster and more intuitive they will become. In natural disasters and 
terrorism-related threat environments filled with uncertainty, however, first responders 
may not have stored schemas they can draw on to help them recognize a situation and 
know how to respond. The development of good SA, as described in the proceeding 
chapter, may help to overcome this experience deficit. 

A. LEVEL 1 SA: PERCEPTION 

Level 1 SA is defined as the “perception and awareness of elements in the 
environment.” 98 This level involves knowledge of surroundings, recognizing familiarities 
along with inconsistencies, and sizing up an incident through observation and surface 
identification. 99 The ability to recognize elements that do not require immediate 
investigation or cataloging typifies an expert operating in level 1 SA. This type of 
situation assessment often results in a familiar “been there before” moment, initiating the 


94 Judith M. Orasanu, “Shared problem models and flight crew performance.” (eds) N. Johnston, N. 
McDonald, and R. Fuller, Aviation Psychology’ in Practice (Aldershot, England: Ashgate Publishing Group, 
1994). 2- 

95 Endsley, “Expertise and Situation Awareness,” 639. 
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Real World,” (eds) Roberta Calderwood, Gary A. Klein, Judith Orasanu, Caroline E. Zsambok, Decision 
Making in Action: Models and Methods (Norwood, New Jersey: Ablex, 1993), 307. 

97 Mica R. Endsley, “The Role of Situation Awareness in Naturalistic Decision Making,” (eds) Gary 
Klein and Caroline E. Zsambok, Naturalistic Decision Making (Mahwah, New Jersey: Lawrence Erlbaum 
Associates, 1997): 270. 
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cognitive process that allows for a decision without proceeding through successive SA 
levels. 100 The experienced decision maker can quickly determine that the situation is 
benign and requires no action. However, there is a down side: An inaccurate assessment 
at this level increases the odds of an incorrect picture fonning. “Studies found that 76 
percent of SA errors in pilots could be traced to problems in perception of needed 
information.” 101 


An analogy can be made to a fire commander’s initial SA size-up at a building 
collapse. The commander performs level 1 SA of this incident by conducting a surface 
reconnaissance of the area, creating a mental blueprint” of existing surroundings. For 
instance, existing voids created from fallen debris may be apparent or perceived; 
however, exploration considerations involve an understanding/comprehension of “risk 
vs. reward” (placing firefighters’ lives in danger for unknown life hazard) consequences 
and are part of the subsequent level of SA. 


B. LEVEL 2 SA: COMPREHENSION 

Level 2 SA involves the comprehension of surroundings, helping the decision 
maker to recognize the relationship of elements identified in the current situation, and 
understand how level 1 SA (perception) affected an individual’s goals and objectives. 102 
For example, it is during level 2 SA that the fire commander will detennine the relevance 
of void exploration by considering defined goals (rescue) in conjunction with 
comprehension of level 1 SA acuity (risk vs. reward). At level 2 SA, novice decision 
makers may experience their first frustration with uncertainty, failing to integrate goals 


100 Endsley, “Expertise and Situation Awareness, ” 639. 

101 Endsley, “Theoretical Underpinnings of Situational Awareness,” 3. 

102 William M. Jones and Mica Endsley, United States Air Force Armstrong Laboratory, “Situation 
Awareness Information Dominance & Information Warfare” (February 1997): 14. 
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and data to achieve comprehension. 103 Although they can conduct an effective level 1 SA 
investigation (perception), they often have difficulty converting perception into 
comprehension to identify the significance of events, and thus are unprepared 
to make good decisions at the following level. 104 

C. LEVEL 3 SA: PROJECTION 

Level 3 SA involves the projection of future events, governed by the decision 
maker’s synthesis of perception acuity and comprehension. Detennination of situational 
elements and their potential impacts upon the environment is imperative at this level. 105 
Knowledge regarding elements of the environment, in conjunction with an understanding 
of the dynamic situation, helps to form level 3 SA. 


The fire commander’s goals/objectives (saving lives / protecting property) have 
been aligned with his current understanding of the environmental elements (unsafe 
operating conditions). Based on this understanding, he believes there is no immediate life 
hazard. His decision to delay void exploration until dangerous, unexplored areas can be 
adequately reinforced with building materials has been made. 


“In almost every field I have studied (aircraft, air traffic control, power plant 
operations, maintenance, and medicine), I have found that experienced operators rely on 
future projections heavily. It is the mark of a skilled expert.” 106 Level 3 SA is the 
commander’s ability to “see over the horizon” regarding current circumstances and their 
relationship to future events. Where is this fire going? Do I have enough resources? Do I 
need additional alarms according to my understanding of the situation? Having the 


103 Endsley, “The Role of Situation Awareness,” 271. 
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knowledge to project future events predicated on dynamic conditions is SA at its highest 
level. “This is analogous to having a high level of reading comprehension as compared to 
just reading words.” 107 

SA can proceed in a progression of awareness from one level to the next or occur 
so quickly that it appears to bypass the comprehension and projection levels and straight 
to intuitive decision making utilizing experience and knowledge, along with predisposed 
schemata stored in long-term memory. Being proficient in perceiving the elements of 
your environment is not sufficient SA. This is rudimentary, conducted on a basic level. 
At this level, a novice and expert may be on a level playing field. The decision maker 
proceeds from level 1 to level 2 SA by successfully linking perception and 
comprehension through considering situational goals. Levels 3 SA captures this synthesis 
while projecting the actions of the elements upon the environment. 108 

SA is an ongoing dynamic process that evolves throughout the course of an event; 
goals are achieved, redefined, or abandoned predicated on situational size-up. A word of 
caution is required concerning the relationship between SA and decision making. Good 
SA does not imply good decision making. An individual can successfully size-up a 
situation and still make a bad choice. Conversely, poor SA can be formed, yet a good 
decision is made nonetheless. 109 The following factors impact the decision-maker’s 
ability to develop good SA and decision-making strategies in complex settings. 

1. Limited Attention and Working Memory 

A decision maker’s limited attention will have a direct bearing on the amount of 
information they can absorb and process. Complex situations, dynamic conditions, and 
the need for multiple tasking are commonplace characteristics of emergency events and 
often result in certain pieces of infonnation receiving more attention than others. The 
more time and attention an individual must devote to infonnation they do not understand 
or have sufficient knowledge of, the greater the possibility of missing a vital piece of 
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information. According to a National Transportation Safety Board (NTSB) review of 
aircraft accident reports, limited attention accounted for 31 percent of accidents involving 
human error. 110 

Limited working memory capacity can inhibit the ability of the decision maker to 
perceive the elements (level 1 SA), comprehend the situation (level 2 SA), and construct 
future projections. A novice with limited working memory or an expert confronting a 
novel situation will experience the same frustrations concerning SA. Lacking previous 
experience, new information and current knowledge must be analyzed in working 
memory; because working memory has a limited capacity it is quickly exhausted, 
resulting in a decision-making process that may be seriously limited. The end result is an 
exorbitant amount of time devoted to conducting problem solving and Level 3 SA 
projections in working memory. 111 

2. Default Values 

Mental models often contain default values for decision makers in the form of 
information that is germane to the particular classification/domain. Default values assist 
the decision process by providing expected characterizations of an element — certain 
understandings that may be applied to the situation in a new manner. Default information 
is useful when the decision maker is dealing in areas of uncertainty while still possessing 
some background knowledge, allowing the expert to continue in situations that often 
stymie the novice. This is true, not because they are experts, but because they possess a 
relevant mental model developed through experience, explaining how a particular system 
works. Default information is helpful in situations where there is missing information or 
overload. 112 Consider the following example. 
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When operating in domains of uncertainty, a specialist trained in hazardous 
material operations (expert) possesses technical knowledge that is germane or applicable 
to many hazardous material situations. At a hazardous material operation, this officer 
knows that chemicals contain particular characteristics (lighter or heavier than air). 
They may not be able to identify the exact agent right away; however, they may be able to 
tell you what it is not. This information leads to actions/options that may have stymied 
the novice or uninformed. 


3. Automatic Action Selection 

Automatic action selection is based on the decision maker’s use of templates 
(scripts) of prototypical situations when making decisions. When operating by using a 
template, less time is wasted on consideration of an alternative plan, thus freeing up 
memory space while allowing the decision maker to process larger chu nk s of information 
and make rapid decisions. 113 Because a response for a particular situation has been 
successful in the past, individuals do not have to search their memory bank. This is an 
example of Klein’s, single-step “recognition-primed” decision model where the decision 
maker recognizes the situation based on comparing cues from the current situation with 
stored templates, and the appropriate response is recognized. The decision maker does 
not need to use the more time-consuming decision process that involves weighing of 
options and evaluating different courses of action. 

While the automatic action selection mechanism provides good perfonnance 
coupled with a low level of attention demand, there may be a down side. Namely, 
because the decision maker is on “automatic pilot,” they may miss a novel situation that 
would normally be recognized. For example, would a person driving a familiar route 
suddenly recognize the insertion of a new stop sign? Probably not, because automatic 
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action selection often results in the decision maker acting on cues (stimulus), short- 
circuiting SA and decision making, and choosing a CO A (response). 114 


World Situation 
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Performance 
of Action 


Stimulus 


Response 


r 


Automaticifcy 


Figure 3. Automaticity (from 115 ) 


4. Categorization of Information 

“Chunking” data allows large amounts of related information to be grouped and 
stored under a single cue. The more developed mental models (as those belonging to the 
hazardous materials fire officer), the more fine-tuned the categorizing can be. In essence, 
if the decision maker possesses an extensive range of mental models, the amount of 
information about the environmental information can be small because it may be that 
only one cue is needed to generate response activity. For example, a small amount of 
information from the environment properly identified in the perception stage can trigger a 
mental model recalling similar situations. 


While operating in the initial stages of a building fire on a lower floor, the roof 
team reports the roof-plumbing vent “hot to the touch ” —a small piece of information. A 
seasoned chief however, knows that plumbing vents are often located in shafts big 
enough to accommodate additional utilities. If fire has entered the shaft (causing the 
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plumbing vent to become hot), it will spread vertically. When the fire reaches the top 
floor via the shaft, it will spread horizontally. Translation: the chief needs help! 


A novice may struggle here because his or her mental model is not as well 
developed, while an expert, although possibly operating in a novel situation, has enough 
critical cues available to stimulate their mental model from past fires and SOPs. 116 

5. Data-Driven and Goal-Driven Processing 

Data-driven infonnation sets in motion SA via a bottom-up decision-making 
process. Salient cues from the environment are instrumental in guiding the decision 
maker for deciding which portions of the environment are attended to (relevant) and 
those that can be ignored (irrelevant). The data-driven process is a bottom-up level 1 SA 
operation. 117 Conversely, the goal-driven progression is a top-down decision-making 
process. Goals and expectations detennine which information is attended to. Synthesized 
data, information, and goals eventually result in the decision maker taking action based 
upon current understandings. A goal-driven process is a level 2 SA operation. 
Throughout SA development, there is a constant interaction between the dynamic and 
developing chain of events (bottom-up) that is indicative of an emergency-type response 
and goal-driven protecting of life and property that often dominates the decision making 
(top-down). While the top-down process will be predicated on existing expectations and 
goals, environmental cues (bottom-up) may require the decision maker to adjust plans or 
activate a new set of goals that align with the current situation. 118 


A fire chief supervising operations at an advanced fire has occupant removal and 
the fire’s extinguishment as command objectives/goals. Because the building is 
structurally sound and still partially occupied, the decision is made to conduct search 
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and rescue operations under the protection of an interior attack hose line. The chief is 
goal-driven, (top-down). A short time later, the chief receives reports that all occupants 
were safely removed from the building and the roof has partially collapsed into the top 
floor. Upon receipt of new information (bottom-up), the chief decides to adjust the 
extinguishment strategy. The chief’s new goals now include a “risk vs reward” 
consideration; interior hose lines are removed and units prepare for an exterior 
operation. 


6. Expectations 

Individuals’ SA can be biased when they become predisposed to certain problem¬ 
solving outcomes based on standing expectations — the if “X’ then “Y” mentality. There 
is a tendency to create a bias between infonnation they are in agreement with while 
ignoring infonnation in conflict with their expectations. 119 Klein calls this process a 
bending of the map.... believing what you want and explaining away any inconsistencies 
that conflict with your expectations. For example: 


Fire extension is prevalent regarding certain types of building construction. 
While a row of attached wood frame buildings may be conducive to fire spread via a 
common cockloft (unobstructed open area between top-floor ceiling and underside of 
roof often connecting two or more buildings), a similar building known as a brownstone 
because it is brick construction is often considered a one building fire because brick 
firewalls extend into the cockloft separating one building from the next. Does this mean 
that fire cannot spread horizontally in a brownstone type building or that all row frame 
structures have common cocklofts? Now consider the receipt of information reporting 
fire in an adjoining building when it contradicts SOP and experience. What role does 
experience have in this situation? 
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When the data conflicts with what you believe, you explain away the 
contradictions. 120 While it is true that individuals will process infonnation faster if it is in 
agreement with their expectations — because they are predisposed in their SA exceptions 
and relying more on working memory — they are also less likely to notice 
miscalculations, 121 not unlike our driver who missed the stop sign because he was on 
“automatic pilot.” 
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III. NATURALISTIC DECISION MAKING 


Decision making in its simplistic form occurs as follows: The individual 
experiences a situation and recognizes relevant cues while discarding those that are 
irrelevant. Recognized cues are processed into patterns that are verified by comparing 
them to mental models. If the pattern and the individual’s stored experiences (mental 
models) are similar, and a “nano-second” mental rehearsal of the action (mental 
simulation) reveals a match, a decision/choice is implemented. 

Research indicates that decision makers in time-compressed situations do not 
typically compare options; 122 there is no comparing of options, trying to decide if one 
choice is better that the other. If the decision or COA selection solves the immediate 
problem, it is implemented. Experience enables the decision maker to size-up the 
situation, determining if it is familiar or atypical, create a match, and implement a 
decision. This intuitive process is completed without deliberation, in a blink of the eye. 
Many first-responder decisions are made in this very manner. Time is a rare commodity 
at a fire scene; decisions are made under conditions of great time pressure, dynamic 
surroundings, and ill-defined goals. 123 


A fire chief is seldom the first unit to arrive on the scene of a developing fire. 
Chiefs are often dispatched upon the receipt of a preliminary report from initial units 
requesting the need for additional resources and command. When the chief arrives, 
drawing from his or her experience, via the use of stored mental models prompts 
anticipation that certain actions have already taken place; for example, hose lines 
stretched, ladders raised, searches in progress, etc. Data driven cues include changes in 
fire conditions as compared to those earlier detected. Namely, earlier reports regarding 
fire extension to floor above via the window is now receding; gray/black smoke is turning 
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white; smoke is observed rising rapidly in a vertical column above the roof. A report 
informs the chief that all victims have been safely removed from the building. 

The chief develops a pattern of cues that is familiar, based on recognized cues, 
while selecting a response that is conducive to an interior fire attack and extinguishment 
efforts based on mental models of past fires. Fire receding in conjunction with smoke 
turning white is indicative of water being applied to the fire; smoke observed rising 
rapidly from the roof of the fire building indicates that a ventilation hole has been cut to 
relieve upper floors of smoke and hot gases while limiting horizontal fire spread. Victim 
removal may no longer be a concern. These actions will make conditions on the upper 
floor favorable for final extinguishment. Cue recognition, pattern matching, and action 
script were all part of the pattern-recognition process behind intuitive decision making. 
There was very little deliberation involved in the decision selection. 


Chapter III identities four major components (cue recognition, pattern matching, 
mental models, and mental simulation) of a naturalistic decision-making process and 
recognition-primed decision model (RPD) and explains their relevance to a first- 
responder’s problem-solving strategy in an asymmetrical threat environment of terrorism 
and natural disasters. 

Naturalistic Decision Making (NDM) is an area of research within cognitive 
psychology that asks how experienced people, working as individuals or groups in 
dynamic, uncertain, and often fast-paced environments, identity and assess their 
situations, make decisions and take actions whose consequences are meaningful to them 
and to the larger organization in which they operate. 124 

The RPD model describes the way a decision maker is able to recognize a 
situation, by identifying typical events and cues and comparing pieces of data with stored 
templates, or schemas of previously experienced situations. Based on the response that 
worked during the previous experience, the decision maker attempts to recognize and 
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evaluate a course of action by often relying on their intuition. 125 Klein defines intuition as 
the way experts translate experience into action. 126 Conversely, novice decision making 
or problem solving situations that are novel (such as asymmetric in nature) may require a 
more structured decision-making strategy, such as mentally rehearsing alternative 
options. 

Chapter III explores this question via a comprehensive description of cue 
recognition, pattern matching, mental models, and mental simulation while supplying 
tangible real world examples (scenarios) that allow the reader to connect theory to reality. 
These scenarios are designed to enhance the reader’s comprehension of a NDM strategy 
through the identification of the recognition-primed decision model (RPD) benefits. 



Figure 4. Recognition-Primed Decision Model. 127 


Prior to developing any plans or procedures for improving first responder 
decision-making skills in an asymmetric threat environment, it is important to define 
decision making terminology and relevance, thereby developing an understanding 


125 Klein, Sources, 24. 

126 Klein, Power, HIV. 

127 Ibid., 26. 


51 



regarding where each component belongs in the decision-making process. For example, 
what is the difference between a mental model and a mental simulation? While mental 
simulation is defined as the process of mentally rehearsing the outcome of decisions, 
determining if an option is workable, 128 mental models are the decision maker’s stored 
experiences, describing how things work. 129 Klein notes that a decision maker’s mental 
model can be enhanced via exposure to numerous opportunities, as in rotating an 
individual through various jobs and duties within an organization until becoming familiar 
with the many functions. 130 Scenario-based training provides another method to build 
mental models. 

Mental simulations are tangible activities in the sense that they involve some form 
of scrutinizing, measuring, and evaluation every time a decision maker weighs an 
option. 131 While mental simulations create a picture or scene within one’s head and allow 
the scene to play out to determine its feasibility, 132 mental models are adaptive beliefs 
that help explain and predict situations. 133 Mental simulations and mental models help to 
explain why some problems can be solved intuitively while others need to be puzzled out. 

The goal for this chapter is to develop the reader’s mental model regarding how 
the decision-making process works and how it applies in everyday first-responder 
problem solving. This will be accomplished by providing related real-world examples for 
each discussed component. One of the main constituents of an effective decision-making 
strategy, regardless of the domain, is the acquisition and nurturing of real-life experiences 
to foster the decision makers’ development of mental models. It is through the construct 
of mental models that the reader will understand the synthesis of the different SA levels 
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of perception, comprehension, and projection. 134 By providing related examples, the 
learner may more readily see the relevance to his particular domain. 


A. CUE RECOGNITION 

Effective cue recognition assists the decision-making process through the 
differentiation of relevant from irrelevant information, immediate from gradual 
transitions, and bottom-up data-driven vs. top-down goal-driven strategies. For example, 
the decision-maker’s ability to discern cues that are relevant and requiring immediate 
action — while considering incident reports and goals — all begin with good cue 
recognition. Cues help the decision maker to construct patterns of behavior, building 
sense-making frames (data importance) that, in turn, activate corresponding mental 
models. 


A neonatal nurse’s intuition interprets an infant’s crying pattern, feeding 
irregularities, distended stomach, and lethargic behavior as critical cues of sepsis, long 
before the ordered blood tests return. Recognizing cues that are relevant, requiring 
immediate action while influencing previous established objectives/goals, is a product of 
prior episodes comprising some or many of the same “tell-tale-signs. ” The nurse has 
“chunked’’ these cues into one universal warning sign. It takes only one symptom (cue) to 
activate a response. 


A fire captain evaluates reports of creaking floors, sagging roof, and leaning 
walls, and detennines that the building is in danger of collapse and needs to be evacuated. 
The captain does not require every critical cue to initiate a response; he probably made 
the decision upon the receipt of any one report. The aforementioned cues prompted the 
captain to intuitively calculate cue relevancy, action requirement, and strategy. 

The more extensive and developed the background and domain-specific 
knowledge neonatal nurses have stored in long-tenn memory, the more efficient they 
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become in responding to emergency situations. How do the previous two scenarios draw 
parallels between cue recognition, SA, and decision making? The nurse and fire captain’s 
recognition of typical from atypical cues, while comprehending their relevance to the 
current environment and implications on future decisions, is a function of level 2 SA. The 
neonatal nurse first perceives the event (level 1 SA) and subsequently develops a strategy 
predicated on comprehension of cue significance relevant to goals and the decision 
maker’s expectancies (level 2 SA). Recognized cues stimulate the recall of information 
from long-term to short-term memory while, at the same time, validating beliefs or 
expectancy. 


While responding to an alarm, a fire chief is attentive to progress reports 
concerning a developing situation. The chief will listen to radio traffic trying to 
determine if the team is “winning or losing’’ at this fire. In this example, the radio traffic 
provides incident cues. Chiefs sense specific strategic and tactical operations because 
cues have stimulated prior experience or mental models; the chief has developed 
expectancies regarding this inciden t. Upon arrival at a fire scene, visual cues such as fire 
volume, smoke color, and direction of fire spread will validate whether the actions the 
chief had anticipated to take place were in fact performed and successful, thereby 
influencing subsequent practical decision making. 


A novice and expert decision maker can view the same event, detect the same 
cues, and yet draw different conclusions. Studies indicate the difference between the way 
an expert and a novice utilize cues is often based more on the amount of weight a 
decision maker assigns to a cue and less on the number of cues considered. 135 They 
consider many of the same cues; however, when processing collected infonnation, 
deciding relevant from irrelevant, the expert may assign a greater weight to one cue over 
the other. This reinforces the theory that experts and novices, when developing their 
perception of an incident (SA level 1), taking in all available infonnation, are equal in 

135 James Shanteau, “How Much information Does An Expert Use? Is It Relevant?” Acta 
Psychologica, 81 (1992). http://www.k-state.edu/psych/cws/pdf/acta_92.PDF. [Accessed July 16, 2007]. 

' 54 



ability. The decision maker is taking the interpretation of cue importance to a new level; 
not only are relevant and irrelevant cues being separated, but relevant cues are further 
distinguished by level of importance or significance, thus helping to create a pattern and 
solve the current problem. Consider the following example. 


An expert and a novice fire officer, conducting a size-up of the same situation, 
will extrapolate three to four significant cues to assist in their development of SA. 
However, the expert will assign a value to collected cues; in essence, the expert is 
beginning to reduce the complexity of the situation by deciding which cue/s is/are the 
most important. In contrast, the novice, with three to four significant cues collected in 
level 1 SA-perception, is often overwhelmed when trying to make sense of this 
information in level 2 SA-comprehension. 


Kemper et ah, comparing the decision-making abilities of experienced and novice 
pilots in time-sensitive situations, reveals that although their reasoning abilities were 
somewhat equal, the experienced pilots excelled because they possess greater domain- 
specific knowledge stored in their long-tenn memory. 136 Experienced pilots recognized 
relevant cues and developed patterns by comparing the current situation with schemas of 
past problem situations. In this study, both novice and expert pilots attempted to find a 
match in long-term memory, but only experienced pilots were able to match cues and 
recall patterns stored in long-tenn memory stores. Conversely, due to limited domain 
knowledge, novice pilots were forced to assimilate cues into short-term memory. Novice 
pilots needed to develop solutions to real-time problems from cues, while problem 
solving in time-pressured, stressful conditions. 137 Results from the Kemper et ah, study 
also indicate that experienced pilots identified more cues that were relevant compared to 
novice pilots who reported fewer relevant clues and more irrelevant ones. Experienced 
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pilots were also more attentive to relevant cues. The best indicator of cue significance 
was probably the amount of time it took to make decisions based upon cue relevance. 138 

Shanteau’s study asking third-year nursing students and faculty nurses to identify 
relevant cues in a nursing scenario revealed that the students’ identification of relevant 
cues was 20 percent higher than the faculty nurses. However, upon taking a course on 
problem solving and decision making, the student nurses reduced their identification of 
relevant cues by 17 percent. 139 The results from this study are encouraging in several 
ways. First, these findings reinforce the critical idea that the difference between an expert 
and novice is not determined by the number of cues they considered, but by the relevance 
of those cues. Second, the fact that the student nurses’ cue recognition skills could be 
improved by attending a problem-solving and decision-making course is inspirational to 
the author’s hypothesis that decision-making skills can be learned by decision makers 
who lack experience. 

This cue filtering process is perfonned in the Level 2 stage of SA-comprehension 
and demonstrates a critical thinking component of the decision-making progression. The 
decision maker is making assumptions and decisions based on intangible information, as 
compared to the corporeal information that can be visualized for authenticity. This 
critical thinking will help the decision maker to think more intuitively when confronting 
this same problem in the future. 

The skills that help experts to solve problems in familiar situations, by utilizing 
cue recognition of similar events to develop pattern matching, are the same skills that will 
help them make good decisions in areas of uncertainty because expert decision makers 
will think more critically about novel situations. Cohen suggests that, because they are 
experts, they have the ability to think critically and, thereby, are more critical of their 
own assessment — and less likely to explain away situations. Because experts are so well 
tuned to cues that foster patterns, they may very well be aware, faster than a novice, that a 
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piece of information is missing. 140 Cohen’s statement regarding experts thinking more 
critically about novel situations is the foundation for this thesis. We know that experts 
may not have the knowledge/experience regarding asymmetric events, because these 
events are rare. Experts do, however, know how to think critically and, hopefully, that 
critical thinking can influence novice decision making. 

The ultimate goal of cue collection and recognition is to take and use the 
information to build patterns of expected behavior while utilizing mental models to 
confirm beliefs. As vital as the collection of cues is to problem solving, if cues cannot be 
transformed and arranged into a coherent pattern of familiarity, they are of little use. 

For example, imagine that while driving along a freeway there is a sudden buildup 
of traffic, with little or no traffic coming from the opposite direction. To an inexperienced 
driver concentrating on the minutia details of the road, staying attentive to their 
immediate surroundings, the pattern of recognition regarding the traffic congestion in one 
direction coupled with a lack of cars in opposing direction may never be comprehended. 
However, an experienced driver recognizes this sudden change in traffic as a potential 
problem ahead (some type of road condition/accident) in their direction of travel or 
possibly in the opposite direction, while observing little traffic coming from the opposite 
direction. These two situational cues help create a pattern of familiarity that is indicative 
of a road problem ahead; namely heavy traffic in one direction and little or no traffic in 
the opposite. 


At this point, the expert’s ability to distinguish between relevant and irrelevant 
information is critical. For example, the novice may consider incident location in their 
decision-making process, while the expert will not. Why? Because the information 
pertaining to incident location has no bearing on the problem at hand. Regardless of 
location, a decision needs to be made involving route selection. For the inexperienced 
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driver, time eventually runs out and they become part of the traffic jam, while the 
experienced driver, without deliberation, exits at the nearest egress and drives the service 
road until past the problem. 


B. PATTERN MATCHING 

“Pattern matching, often performed intuitively, refers to the ability of the expert to 
detect typicality and notice events that did not happen and other anomalies that violate 
the pattern.” 141 Problem solving by the utilization of their pattern-matching skills, in 
conjunction with intuition and experience, is often a first responder’s decision-making 
modus operandi. 

When dealing with uncertainty, decision makers begin to identify irregularities 
(via cue recognition) based on their expectations regarding previously experienced or 
known patterns of behavior. Developed recognition provides patterns for future 
decisions. 142 This is imperative because events are more easily recognized and re-created 
when their structure makes sense or a decision maker has domain-specific knowledge. 
The more expertise an individual has acquired in a particular domain, the more elaborate 
the portfolio of experience at their disposal for comparison and greater recognition. If 
they are not experts or experiencing a novel asymmetrical situation, they may recognize 
cues; they will struggle, however, in constructing and detecting typical patterns. 


For example, when a group of chess masters was asked to re-create the positions 
of observed chess pieces on a board, they were better able to re-create those positions 
that followed a logical sequence of chess moves and positions than those that were 
randomly placed on the board. 143 What this tells us from an asymmetrical threat 
environment perspective is that while a fire chief can mentally recreate hose line 
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positions, fire apparatus placement, and incident command post locations while 
responding to a particular type of fire, he may not be afforded this luxury at a hurricane 
or dirty bomb explosion. In addition, the individual will spend an excessive amount of 
time in working memory, creating meaning from knowledge of surroundings 
— solving problems but missing critical cues. 


Ericsson et al., report that experts are only experts when solving problems in their 
own domain. For example, a seasoned urban fire commander may not be an expert when 
operating at a rural forest fire. The risks associated with experts operating in novel 
situations include the use of inappropriate patterns of recognition from past events. While 
pattern matching is designed to assist in the decision-making process, the decision maker 
may ignore inconsistencies to make the pattern fit . 144 This process was cited earlier as 
Klein’s “bending of the map.” The decision maker explains away inconsistencies to 
justify an initial pattern match with what has worked in the past. The rationalization of 
anomalies and inconsistent patterns in the decision-making process only compounds the 
situation, often reaching a point of no return . 145 Revisiting the neonatal nurse’s decision¬ 
making scenario, consider the following regarding pattern matching, intuition, and 
experience. 


The critical role of recognition in decision making came into sharper focus when 
Beth Crandall, 51, vice president of research operations at Klein Associates, got a 
contract from the National Institutes of Health to study how intensive-care nurses make 
decisions. In 1989, she interviewed nineteen nurses who worked in the neonatal ward of 
Miami Valley Hospital in Dayton, Ohio. The nurses cared for newborns in distress — 
some post mature, some premature. When premature babies develop a septic condition or 
an infection, it can rapidly spread throughout their bodies and kill them. Detecting sepsis 
quickly is critical. Crandall heard dozens of stories from nurses who would glance at an 
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infant, instantly recognize that the baby was succumbing to an infection, and take 
emergency action to save the baby's life. How did they know whether to act? Almost 
always, Crandall got the same answer: "You just know." 146 


Crandall asked the nurses to recall the details that lead them to believe the infant 
was suffering from sepsis. Complexion fade from pink to gray, changes in feeding 
patterns, swollen abdomen, and a frequently crying baby that suddenly becomes listless 
and lethargic were among the symptoms that helped to develop a pattern. A novice nurse 
could easily explain each of these symptoms when they occur in isolation; however, an 
experienced nurse chu nk s symptom together to form a pattern, draws upon past events 
(experience) concerning infants diagnosed with sepsis who exhibited not all but many of 
the same symptoms, and intuitively creates a match. When infants display enough 
symptoms to develop a pattern indicative of sepsis, the nurses know what to do. Many 
times the baby is placed on antibiotics days before blood tests return confirming sepsis. 
“Each of these cues is extremely subtle but, taken together, they are a danger signal to an 
experienced nurse.” 147 This is why the development of pattern recognition skills is so 
important to decision making. 

Once the decision maker recognizes a pattern as familiar, intuition activates action 
scripts. We intuitively kn ow what will work predicated on experience. While intuition 
has been described as a “gut feeling”, it may be better described here as a not conscious 
thought process because the person has seen/experienced it before and they automatically 
recognize it and can act based on recognition of the situation. We sense what to expect 
next while recalling routines (action scripts) for responding. The more patterns/action 
scripts a decision maker has, and the more expertise they possess, the easier is to make 
decisions. 148 Consider the following examples of the pattern recognition process to 
intuitive decision making: 
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Fire ground commanders have different action scripts for each situation. One 
script may address search and rescue while another pertains to fire confinement and 
exposure protection. The fire commander constructs a set of patterns, generated from 
cues, organizing information particular to their domain. These patterns are stored in 
long-term memory. 149 The decision maker’s experience level, pattern recognition skills, 
and developed associations between patterns and action scripts are all influential factors 
concerning a fire commander’s decision making in a pressured environment. 150 


Using pattern recognition in critical time sensitive situations, matching situational 
cues and patterns with automated response actions appears to be the way that firefighters, 
nurses, and military commanders make split second decisions. Intuitive decision making 
is predicated on effective pattern matching. 151 However, when the data are conflicting 
and unreliable, when information is incomplete or ambiguous, how do these same 
individuals perform situational assessments? 

C. MENTAL MODELS 

Experienced decision makers have internal representations of the system they are 
dealing with. They refer to these representations as mental models. 152 “Mental models 
are adaptive belief constructs used to describe, explain, and predict situations.” 153 
Developed from years of practice, a mental model can be viewed as a mental outline of 


'49 Crandall et al., “Training Decision Makers for the Real World,” 314. 

150 Ibid., 312. 

151 Marvin S Cohen et al., “Critical Thinking Skills in Tactical Decision Making: A Model and a 
Training Strategy, “Making Decisions Under Stress (eds). Janis A. Cannon-Bowers and Eduardo Salas, 
(Washington, DC: American Psychological Association, 1998): 156. 

152 Endsley, “The Role of Situation Awareness,” 274. 

153 Kevin Burns. “Mental Models and Normal Errors,” How Professionals Make Decisions (eds), 
Berndt Brehmer, Raanan Lipshitz, Henry Montgomery (Mahwah, New Jersey: Lawrence Erlbaum 
Associates, 2005): 17. 
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expected events for a dynamic situation. 154 Because a decision maker’s working memory 
capacity is limited, mental models stored as long-tenn memory representations are 
retrieved to depict and predict how the world operates. 155 They allow the decision maker 
to use long-term memory stores to solve present day problems, attending to the dynamic 
unfolding circumstances of the event at hand. 

A mental model’s domain is comprised of features, functions, and 
applicability. 156 For example, an individual becomes familiar with a particular piece of 
equipment (tool). They know the tool’s features, capabilities, and components. They 
know where and when the equipment is used and how the tool will react with the 
environment. When employing this piece of equipment, the decision maker compares the 
current situation with his mental model of the equipment’s capabilities and predicts its 
effectiveness. In essence, a parallel is created, comparing the past (memory) with the 
present situation. If there is a match, the decision maker overlays the mental model on the 
situational model. If not, he takes additional steps (options) to solve the problem. 

First responders working in similar domains for extended periods of time, 
responding to specific types of events, develop mental models depicting how certain 
events should evolve, the way things work — seeing inside the object or event. A 
hazardous material technician becomes an expert in the handling of hazardous material 
incidents because he specializes in responses of this type. Technicians are often 
summoned to mitigate hazardous material responses that are beyond the technical 
capability of operating first responders. By repeated exposure to domain-specific events, 
technicians develop and store experiences — mental models from previous operations. 
They become experts. 


154 Andrew R. Dattel and Francis T. Durso, “Expertise and Transportation” (eds) Neil Charness et al., 
The Cambridge Handbook of Expertise and Expert Performance (New Y ork: Cambridge University Press, 
2006): 366. 
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A fire chief assigned to a high-rise building district for an extended period of time 
should begin to develop mental models to aid in strategic planning and decision making 
when responding and operating at high-rise building fires. How well these buildings 
resist vertical and horizontal fire spread, along with their vulnerability to collapse, will 
become the subject of numerous after-action reports and fire critiques. The chief’s 
success stories, failures, and near misses will be fodder for mental model development. 
Decisions that worked, fell short, or were classified as last option choices will be filed 
away for future retrieval. The chief becomes sensitive to any information that may trigger 
a response action. His mental models are extensive enough to filter information, deciding 
relevant from irrelevant, identifying which information needs his immediate attention. 
When confronting incidents of a similar type, the fire chief relies on prior knowledge or 
mental models of how the world operates according to prior experience. 


It would seem logical to assume that the more time individuals spend in particular 
domains, the more numerous and extensive their mental models will become. In essence, 
they are becoming experts in their domains. Conversely, Klein suggests the rotation of 
employees through the different jobs of a corporation, exposing them to new and 
challenging situations and job requirements, thereby enhancing their mental models. 
When required to work with the different entities of a corporation, each employee will 
have greater knowledge concerning job characteristics. Consider the following example 
of mental model building. 


In an attempt to expand a firefighter’s experience/knowledge base concerning the 

wide-range of building construction and fire related scenarios a firefighter might 

confront, the Fire Department City of New York (FDNY) established a training program 

requiring newly promoted firefighters to work in several firehouses and locals 

throughout their first three years of service. The average rotation time spent in each 

firehouse was approximately one year. This was groundbreaking considering the fact that 

many firefighters work their entire career in one firehouse, becoming resident experts 
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regarding buildings or area oddities specific to their response district. For instance, 
firefighters dedicating their entire career to a low-rise building district, becoming quite 
proficien t concerning fire operations in buildings germane to their district, may be at a 
distinct disadvantage if they suddenly find themselves working in mid-town Manhattan 
amongst the forty- or fifty-story office buildings. By rotating through several firehouses 
scattered throughout the city, the firefighters have the opportunity to experience fires and 
emergencies that are not predicated on a particular domain. The goal was to produce a 
more rounded firefighter with varied mental models. 


Mental models assist decision making in two distinct ways. First, when we use 
mental simulations to puzzle out problems, mental models help to predict outcomes. 
Second, mental models help the decision maker to recognize inconsistencies when pattern 
matching the present to the past, recognizing what is typical and what is out of the nonn. 
Mental models are a type of story building; they give us a casual account of how things 
work while helping us make educated guesses and spot inconsistencies. 157 

Klein et al., take the mental model theory to the next level by identifying and 
distinguishing the difference between comprehensive models and just-in-time (JIT) models. 
Comprehensive models belong to individuals who possess thorough knowledge in a 
particular domain. For example, mechanics who work primarily on diagnosing and repairing 
automobile transmissions would possess comprehensive models in transmission repair. A 
transmission-related problem activates long-term memory stores that retrieve mental models 
of past transmission repair work. When the job is successfully completed, the results are 
stored in long-term memory for future use. 

“In complex and open systems, a comprehensive mental model is unrealistic. ”158 
While the comprehensive mental model is static, first responders often needdynamic, 
fragmentary knowledge to help make sense of a situation. This sense making is the prelude to 
JIT mental models. 159 


157 Klein, Power, 314. 

158 Klein et al., “A Data-Frame,” 151. 

159 Ibid., 130. 
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JIT mental models rely on the belief that fragmentary knowledge from one 
domain can be applied to problem solving in another domain. When decision makers 
discover themselves troubleshooting, working with limited knowledge, considering 
choices without a complete understanding about what is going on, they often utilize JIT 
mental models as an additional method to problem solving. The concept of JIT is in no 
way related to time constraints, as if the JIT model was devised at the last possible 
second. Instead, it is a theory whereby the decision maker, because there is no 
comprehensive mental model, will construct a mental model at the time it is needed. 160 A 
dividend of the JIT mental model process may be the attainment and storage of new 
knowledge in long-term memory. 

There are domains where the need for a comprehensive type of mental model 
exists; for example, a cardiac system is somewhat closed and refined as science 
progresses. However, compare this to the fire chief standing in front of a burning 
building, developing and creating multiple JIT models to make effective decisions. 

A fire officer will rely on knowledge/experiences acquired from similar situations 
and develop a COA based on what has worked in the past. For example, in an earlier 
scenario, while processing reports of sagging floors within the building, the fire captain 
recalls a similar situation from long-term memory when reports of sagging floors were 
followed by a building collapse. Imagine the fire captain receiving relevant information 
regarding individual safety but minus the mental model that provides a COA or the 
ability to construct a JIT mental model. Translation: The fire officer does not need to be a 
building engineer to possess enough fragmentary knowledge regarding indications of 
building collapse. 

Experts are making better on-the-spot decisions because they have a greater 
fragmentary knowledge base, a more thorough cause-and-effect relationship than their 
counterpart who is considered a novice. Klein et ah, suggests that we train people by 
expanding their “repertoire and depth of casual relationships, rather than trying to teach 


160 Klein et at, “A Data-Frame,” 131. 
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fully worked out mental models within a domain.” 161 This statement is encouraging and 
reinforces the belief that SA, decision making, and critical thinking templates can be 
developed for today’s asymmetrical threat environment. Consider the example and how 
mental models contributed to an intuitive decision-making scenario. 


It is a simple house fire in a one-story house in a residen tial neighborhood. The 
fire is in the back, in the kitchen area. The lieutenant leads his hose crew into the 
building, to the back, to spray water on the fire, but the fire just roars back at them. 
“Odd, ” he thinks. The water should have more of an impact. They try dousing it again, 
and get the same results. They retreat a few steps to regroup. Then the lieutenant starts to 
feel as if something is not right. He does not have any clues; he just does not feel right 
about being in that house, so he orders his men out of the building — a perfectly standard 
building with nothing out of the ordinary. As soon as his men leave the building, the floor 
where they had been standing collapses. Had they still been inside, they would have 
plunged into the fire below. 162 Table 3 identifies and connects the different RPD 
componen ts of the simple house fire scenario. 


161 Klein et al., “A Data-Frame,” 132. 

162 Klein, Sources, 32. 
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Table 3. Simple House Fire 


Cues recognition 

Pattern matching 

Mental models 

One-story house in a residential 
neighborhood 

Bedrooms/living areas on first 
floor or basement 

Extinguishing fire allowed units 
to search the first floor/basement 
bedrooms 

Fire is in the back, in the kitchen 
area. 

Food on stove, gas-fueled fire, 
oven fire, electrical fire, grease 
fire 

Food on stove became 2 nd alarm 
fire — there are NO routine fires 

The water should have more of 
an impact 

Water extinguishes fire and 
reduces heat; black smoke turns 
white 

Gas fueled fire required the gas to 
be turned off before final 
extinguishments. 

Kitchen fire that was an extension 
of a basement fire. Voids provide 
undetected extension of fire from 
basement 

Dousing it again, and get the 
same results. 

Additional water should 

accomplish final extinguishment 

Accelerant fire with inadequate 
water pressure. 

Perfectly standard building with 
nothing out of the ordinary 

Sound buildings not subject to 
collapse 

Basement fire burned undetected, 
and caused first floor to collapse; 
water did not reach seat of fire. 


When dealing with an asymmetrical threat environment, first responders will 
often not possess a mental model for the situation at hand, nor the capacity for engaging 
in pattern recognition, and must therefore have a contingency plan. Mental simulations, 
rehearsing mentally how a selected option may play out and comparing different courses 
of action, is one such step. 

D. MENTAL SIMULATION: 

Mental simulation is defined by Klein as “the ability to imagine people and 
objects consciously and to transform those people or objects through several transitions 
finally picturing them in a different way than at the start.” 163 

Mental simulation comes into play when the “simple-match RPD model” fails to 
intuitively recognize a solution. Teaching the art of mental simulations involves mentally 
rehearsing options to determine workability for current problem solving, inputting, or 
adjusting as warranted. The development of good mental simulation skills will improve 
accuracy and save time in critical situations. 


163 Klein, Sources, 43. 
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By rehearsing a COA, mental simulation helps the decision maker to make sense 
of events that are void of the critical cues and patterns that activate intuition. 164 The 
decision maker will play out options in their mind's eye, making use of mental models, 
analyzing actions that must be taken, and results of implementation, while identifying 
problems with selected courses of action/options and their solution. 165 “When we use 
mental simulation to derive a plausible explanation, we feel that we have diagnosed the 
situation the way a physician might diagnosis a disease or an auto mechanic might 
diagnose an engine problem.” 166 


A fire commander responding to an early morning alarm for an advanced fire in a 
newly constructed commercial building must decide between an interior or exterior 
attack. Among the many cues the commander will consider in his size up, two or three are 
critical and are immediately apparent; the amoun t and volume of fire, time of day, and 
building stability. Before making a choice, the commander will consider relevant cues, 
recognize patterns, and compare and contrast the patterns to mental models of previous 
fires. If time permits or patterns are not making sense, the commander may mentally 
rehearse a series of actions predicated on cues. For example, the volume of fire could 
indicate how long the building has been burning or if the fire is perhaps arson related; 
commercial buildings are normally unoccupied after business hours; and newly 
constructed buildings are not built to burn, while those in the state of disrepair are prone 
to collapse earlier than expected. 


Regarding the commercial building fire scenario, remove any information 
indicating that the fire is in the advanced stage. For some fire commanders (experts) the 
remaining two critical cues (time of day and construction) may still be enough to make an 


164 Klein, Sources, 89. 
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intuitive decision in favor of an exterior attack. The commander’s mental simulations 
concerning decisions made can be cognitively processed in numerous ways. First, if time 
pennits in an intuitive setting, the commander may mentally rehearse the decisions to 
identify any unforeseen problems before implementing action. He may modify or reject 
an option or possibly explore another action. 167 Second, when a problem or a decision¬ 
making opportunity does not provide enough familiarity to be solved intuitively, the 
commander will decide on a COA by mentally playing out identified options to 
determine their feasibility for success. If one works, he will implement the decision; if 
not, he will choose another course of action and repeat the simulation steps. 

Continuing with the previous scenario, imagine that the commander is 
considering suggestions to implement an interior attack. Because there are no mental 
models available for pattern matching, he will mentally play out how the process will 
unfold. He may envision the following: 

• Optimum apparatus placement for unit safety. Is it available? 

• Proximity of command post to building for adequate communication with 
interior operations. Can communication be maintained? 

• Resources on hand if plan needs to be changed. How fast can they go to an 
exterior attack if needed? 

By mentally rehearsing options, the commander will validate or disprove their 
COA. Agreed upon decisions are not necessarily the best choice, although they are a 
workable solution. This is because, in many highly time-compressed decision-making 
situations such as firefighting, there is not time to come up with the optimal solution. 
Instead, decision makers use a “satisficing” strategy to arrive at a workable solution given 
the time constraints. 168 


167 Gary A Klein, “A Recognition-Primed Decision (RPD) Model of Rapid Decision Making,” 
Decision Making In Action: Models And Methods, (ed) Roberta Calderwood et al, (Norwood, New Jersey: 
Ablex Publishing Corporation, 1993) 140. 

168 Stanley C. Collyer and Gerald S. Malecki, “Tactical Decision Making Under Stress: History and 
Overview,” (eds) Janis A. Cannon-Bower and Eduardo Salas, Making Decisions Under Stress 
(Washington, DC: American Psychological Association, 1998): 7. 
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A word of caution concerning mental simulation: The person constructing the 
mental simulation needs to be familiar with the particular task, thinking at the right level 
of abstraction. You cannot mentally simulate an option foreign to your domain of 
expertise. For example, a fire chief has no background knowledge regarding the mental 
cognitions of a neonatal nurse rehearsing emergency interventions of a newborn in 
distress. 

Mental simulations do not need to be elaborate. Due to the limits of our working 
memory, six steps should be the limit. If there are too many transitions, the decision 
maker will lose track. 169 When span of control becomes an issue, the decision maker can 
“chu nk ” several transitions into one unit. If the simulation is too detailed, it will take up 
critical working memory space. The decision maker can assume certain steps being taken 
without actually thinking it out. This presumption is a good example of SA automatic 
action selection. A simulation comprised of intangibles cannot be measured, therefore a 
certain amount of domain knowledge is necessary to separate fact from fiction. 170 

The decision maker must keep in mind that simulations can be wrong; “they are a 
means of creating explanations, not for generating proofs.” 171 While mental simulation is 
effective when the decision maker is not quite sure what is happening, trying to puzzle 
out the problem; time-pressure, and too many moving parts can complicate their puzzle 
and affect the quality of mental simulation. 172 

In summary, filtered information from the environment activates the mental 
model, creating numerous situational models designed to assist decision making. 
Selective cues help the decision maker to recognize patterns, activating action scripts in 
the fonn of mental simulations. Mental simulations rely on stored mental models to 
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mentally rehearse different scenarios to detennine if the decision maker’s choice will be 
effective. “In order to build an effective mental simulation, we need to have good mental 
models of how things work.” 173 


173 Klein, Power. 27. 
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IV. DECISION-MAKING STRATAGEM 


Chapter IV outlines a stratagem to transform first responders into better decision 
makers when confronting uncertainty. Earlier chapters described SA and the RPD model 
strategies of cue recognition, pattern matching, mental models, and mental simulations. 
Chapter IV, however, identifies the characteristics that constitute an effective decision¬ 
making strategy in the following ways: first, through identification and evaluation of the 
mechanisms of NDM and expertise; second, by categorizing trainable decision-making 
skills and the methods for training them; and finally, by suggesting a scenario-based 
experiment to detennine if first-responder decision-making perfonnance in an 
asymmetrical threat environment can be enhanced. 

NDM strategies enhance an individual’s intuitive decision-making skills, while 
SA fast tracks proficiency and expertise development for a novice decision maker’s 
training. While training has been and will continue to be considered the educational 
methodology of choice, intended to enhance decision making, it may not be the 
appropriate word to describe this process. Two issues are paramount regarding a person 
becoming more skilled in their domain: First, educators may want to consider the process 
more of an education than training regimen; “training provides specific skills, whereas 
education provides the basis for further skill acquisition.” 174 Second, Klein tells us that 
training a novice to think like an expert is too time-consuming and expensive. However, 
maybe we can get a novice to learn like an expert. 175 As discussed earlier, a novice can 
learn like an expert by engaging in deliberate practice, compiling an extensive experience 
bank, and obtaining accurate feedback on decisions made. 176 

Before beginning an educational undertaking designed to prepare novice decision 
makers to leam like experts in asymmetrical threat environments, the “end to the means” 
regarding the characteristics that underlie effective decision making must be detennined. 

174 Stoney A. Trent, “Team Cognition in intelligence analysis training,” Dissertation (Ohio State 
University, 2007): 21. 

175 Klein, Sources, 104. 
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In other words, what should they be able to do? While there are numerous skills and 
training methods for enhancing decision-making effectiveness, the following table 
outlines the characteristics (knowledge and skill) and mechanisms of NDM that enhance 
decision-making performance in an asymmetrical environment — those decision-making 
skills and methods specific to this study. By doing so, educators will be better prepared to 
make the transition from “what to train” (skills) to “how to train it” (methods). 


Table 4. NDM Characteristic and Mechanism/Expertise 


Characteristics 

Mechanism/Expertise 

• Flexibility/adaptability 

• Quick/resilient 

• Risk-taker/accurate 

• Situational-Awareness skills 

• Organized knowledge structure 

• Mental simulation, models, cue/pattern 
recognition 

• Reasoning skills 


A WHAT TO TRAIN (SKILLS) 

Situational Awareness Skills 

• Ability to conduct a rapid accurate assessment; synthesis of perception, 
comprehension, and projection SA Levels. 

Cue recognition/pattem-matching skills. 

• Trusting intuitive decision making 
Mental Simulation Skills 

• The art of rehearsing a solution 
Mental Models 

• Selecting the right model for the right situation/constructing templates 
Domain-Specific Problem-Solving Skills 

• Problem solving in domain of expertise, not generic 
Reasoning 

• Critical thinking and creative problem solving/improvising 
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B. HOW TO TRAIN (METHODS) 


Design scenario based training, i.e., computer based simulations, paper/pencil 
decision skill training, tabletop exercises... 

• Domain-specific scenarios allow the trainee to buy into the hypothesis of the 
situation. If individuals can see the relevance of the training to their job, learning 
will take place. “The most important criteria concerning simulations are whether 
the decision maker takes the scenarios seriously.” 177 

• Scenario-based training can be the conduit for exposing individuals to the type of 
situations they will confront in real-world settings. Unlike real world events, 
however, and full-scale exercises, scenarios can be stopped, paused, and rewound. 
Mastered skills can be fast-forwarded, while remedial training can be conducted 
on those skills not yet perfected. For example, scenario simulations are safer than 
full-scale exercises, not exposing individuals to unnecessary risks. 

• Scenario-based simulations provide more practice trials; the very problem-solving 
skills that the trainer is trying to enhance — namely situational awareness, cue 
recognition, pattern match, mental models, and mental simulation — are 
strategically included in the scenario. These skills can be enhanced through 
repeated exposure (deliberate practice) and designing scenarios around training 
objectives. For example, displaying cues with associated patterns will help to 
develop templates (mental models) for future use. 178 Because opportunities to 
develop decision-making expertise via real-world events are limited, scenario- 
based training provides a chance to acquire skills that real-world events may not 
have the opportunity provide. 

When considering “what to train,” educators need to make certain the 
instructional design is context-specific domain knowledge. To make training truly 
effective, individuals need to work within their area of expertise. For example, if the goal 

177 Gary A. Klein and David D. Woods, “Conclusion: Decision Making In Action” (eds) Roberta 
Calderwood et al., Decision Making In Action: Models And Methods (Norwood, New Jersey: Ablex 
Publishing Corporation, 1993): 410. 

178 Canon-Bowers, “Training Decision Makers for Complex Environments: Implications of the 
Naturalistic Decision Making Perspective.” 105. 
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is to enhance a fire officer’s decision-making capabilities while operating in areas of 
uncertainty, scenario-based instructions should begin by introducing the officer to an 
environment comprised of familiarities, one in which they feel somewhat comfortable 
regarding incident problem solving. Then, once familiarity is established, strategically 
tweak the scenario with situational injects that develop uncertainty. This is an ideal 
opportunity to evaluate JIT mental models — the use of fragmentary knowledge to 
activate good mental models. 179 Do novice officers possess enough domain knowledge to 
enable them to piece together a good synopsis regarding the uncertainty? Chapter V 
explains the scenario-based training methodology, while describing the thesis experiment 
in detail. 


179 Klein et at, “A Data-Frame,” 130. 
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V. SCENARIO-BASED LEARNING AND 
EXPERIMENTAL DESIGN 


We have discussed scenario-based examples as a means to provide exposure to 
asymmetric scenarios that first responders do not experience in their normal jobs. 
Scenario-based training provides an effective way to expose practitioners to events and 
let them respond when there are not opportunities in every day practice. The author’s 
goal here is to reinforce the decision maker’s thought process by creating a skill and 
application connection. 

Scenario-Based Training (SBT) entails the development of cue-driven exercises. 
The objective is to expose trainees to response environments similar to those in their 
professional lives, while providing feedback on responses. According to Cannon-Bowers 
et ah, SBT helps accelerate the learning process for the decision maker from novice to 
expert status by providing deliberate practice with a series of task-related episodes. 
Deliberate practice builds familiarity. SBT also provides trainees with an opportunity to 
work with equipment that is often not readily available in a controlled laboratory 
setting. 180 

According to Ayers et ah, SBT follows a perfonnance-based imperative. It is 
different from traditional training because scenarios incorporate a specific decision¬ 
making component. 181 SBT must focus on the outcome and how it can be improved, 
rather than on the transfer of skills or the acquisition of knowledge. While skills are 
imperative to SBT, they are attained via traditional training and through experience. 
Scenario-based exercises immerse the decision maker in situations requiring 
performance. 

180 John J. Bums, Jams A. Cannon-Bowers, Eduardo Salas, John S. Pruitt. ‘‘Advanced Technology in 
Scenario-Based Training” (eds) Janis A. Cannon, Eduardo Salas, Making Decisions Under Stress. 
(Washington, D.C.:American Psychological Association, 2006): 365. 

181 Frank Ayers, MS, Thomas Connolly, Ed.D, Charles Robertson, Ph.D. Michele M. Summers, MS 
“Managing Risk through Scenario Based Training, Single Pilot Resource Management, and Learner 
Centered Grading” (September 2007): 8. 
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SBT is the next best thing to reality; “Reality is the ultimate learning situation and 
scenario-based training attempts to get as close as possible to this ideal.” 182 SBT 
incorporates effective real-life scenarios coupled with expert feedback, providing an 
opportunity for a novice to learn like an expert. “In other words, we leam better the more 
realistic the situation is, and the more we are counted on to perform.” 183 Washington 
State homeland security stakeholders, when asked for their suggestions on improving 
training materials, cited real life and recent examples wherever possible. 184 According to 
the Homeland Security Institute Report to the WA State Committee on Homeland 
Security, the most difficult challenge regarding training is relating knowledge from 
training to “real life.” Real-life scenarios help participants apply what they know; two by¬ 
products of this application are the use of judgment and skill. 

A. EXPERIMENT HYPOTHESIS 

Through the utilization of scenario-based instruction, this experiment attempted to 
determine whether the decision-making skills of newly promoted fire officers could be 
enhanced through specific training designed to develop those skills. The training 
consisted of practice with asymmetric terrorist and natural disaster scenarios, expert 
feedback, and a brief lecture on SA / RPD model strategies. 

By manipulation of the independent variables (SA / RPD brief and expert 
feedback), the author attempted to achieve the desired outcome of improving novice 
decision-making skills (dependent variable). In other words, feedback and SA / RPD 
reinforcement (independent variables) should have improved novice decision makers’ 
problem-solving and decision-making skills in the final scenario. The end goal was to 
determine if novice decision-making skills could be enhanced, thereby training them to 
learn like an expert. 

182 Ayers et al., “Managing Risk through Scenario Based Training.” 

183 Ibid. 

184 Emergency Responder Training Assessment and Recommendations,” Homeland Security Institute 
Report to the WA State Committee on Homeland Security (December 2005): 14. 
http://www.metrokc.gov/prepare/docs/2006-02- 

23%20001%20Emergency%20Responder%20Training%20Assessment%20and%20Recommendations.pdf. 
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B. 


GROUP DESIGN 


Three groups were assembled: control, expert, and experimental. The experts 
were fourteen New York City Fire Department officers with extensive job knowledge and 
decision-making experience, identified by their current rank, years of service, and self- 
reported domain expertise. The experts had a minimum of fifteen years of fire department 
experience and at least one year in their current rank. The experimental and control 
groups consisted of twenty-nine New York State Fire Officers promoted within the past 
twelve months. While the use of a random lot selection would ensure that both groups 
were of equal distribution, group designation (experimental/control) was detennined by 
the availability of participants. The control group consisted of fifteen officers from 
various fire departments located throughout New York State, excluding the FDNY. The 
experimental group was fourteen recently promoted FDNY lieutenants. While the period 
of time spent in previous rank (firefighter) varied, officer field-experience for the 
experimental and control groups was limited or in some cases non-existent. 

C. PRE-EXPERIMENT QUESTIONNAIRE 

Prior to exercise commencement, each group completed a brief questionnaire 
giving greater insight into their fire service experience and additional domain-specific job 
knowledge. The expert’s mean score of years of service with the fire department (25) was 
two times more than the experimental (9) and control (12) groups, while experts years in 
current rank (5) was five times higher. In addition to confirming their status as an expert 
or novice, information was collected on their familiarity with SOPs. The pre-experiment 
questionnaire asked groups to rate their knowledge regarding fire department standard 
operating procedures (SOP) at certain types of responses, while avoiding the risk of 
predisposing subjects to certain assumptions, namely a terrorist attack. For example, a 
terrorist-related scenario depicts an explosion at a subway station, the questionnaire asked 
about subway/transit SOPs and not about an explosives or terrorism-related response. The 
experts rated themselves higher than the experimental and control groups in the five- 
knowledge/experience categories. The below average scores of the experimental and 
control group solidifies their status as novice decision makers (see Appendix A). 
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D. 


THESIS EXPERIMENT 


The experiment began by providing the expert group with the one-page brief 
describing SA and RPD strategies. This description established a connection between 
decision-making theory and expert practical application by providing real world 
analogies (see Appendix B). Next, a written practice scenario depicting an emergency 
response to a novel situation, i.e., a terrorist scenario, was distributed to each member of 
the expert group. The researcher read the scenario passage aloud to a selected decision 
point (participants could read along silently). At each decision point, the experts were 
asked one to three questions in several different formats. For example, one format asked 
the expert group to prioritize from a list of possible options, while another fonnat asked 
the group to provide a yes or no response. The lists of possible actions/options to decision 
point questions were developed from five pre-experiment scenario pilot sessions with 
other peer experts. 185 

A scenario response key was developed by considering the experts’ most common 
responses as correct answers to decision point questions. For example, if twelve out of 
fifteen experts provided the same response to a question, that response was considered 
correct. For responses requiring experts to prioritize from a list of six to seven actions, 
the top three prioritized responses became the answer key. The experts were also asked to 
explain their decision-making process and problem-solving strategies in the context of 
SA and RPD, the why behind their decision making and prioritization. These responses 
provided the foundation for developing feedback for the experimental group. For 
example, an underlining theme throughout the scenario readings was the protection of 
first responding, making sure they do not become part of the problem by becoming 
victims themselves. One expert provided the following analogy to reinforce the need to 
protect first responders first: 

185 At the pre-experiment sessions, the scenarios were read to other experts in the same fashion. Their 
responses were captured, summarized, and studied for commonalities between group members. Common 
responses were developed into actions or decision-making option lists unique to each scenario decision 
point questions. These pilot-sessions turned out to be invaluable during the actual experiment because the 
groups were given 2.5 minutes to make decisions and provide explanations when asked. By asking the 
individuals to prioritize from a list of responses designed solely to that specific passage, all three groups 
were considering the same information and suitable responses. 
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Part of an airline pre-flight safety plan prior to take-off includes instructions regarding 
the donning of individual breathing masks — in case cabin pressure is lost while the 
plane is in the air. Individuals with a child are told to don their mask first and then the 
child’s. Why? Because, if conditions require supplementary oxygen, you may be 
overcome while trying to outfit your child. In which case, what good would you be to 
your child? The same analogue goes for firefighting. Managers need to make decisions 
ensuring their personnel are properly equipped before sending them into danger ... we 
must protect ourselves first ... and this involves decision making ... possibly unlike those 
of the past because ofprior knowledge/experience. 


This procedure continued until all the scenario passages were read and scenario 
decision points completed. A total of four asymmetric type scenarios (tanker on bridge, 
subway explosion, radiological attack, and earthquake) were read to the expert group, 
with three practice and one actual experimental test scenario (see Appendix C, D, E, F) 
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Figure 5. Expert Group Experiment Matrix 

The same scenarios were distributed to the control and experimental groups. Each 
scenario passage was read aloud to the decision point (participants could read along 
silently). At each decision point, the researcher asked one to three questions and then 
gave participants 2.5 minutes to provide a response. This procedure continued until all 
the scenario passages were read and scenario decision points completed. 

Upon completion of each scenario decision point, the experimental group was 
exposed to the independent variables (SA / RPD brief and experts’ feedback). The control 
group was not exposed to the independent variables. The goal of expert feedback was to 
improve the decision making of the experimental group at subsequent scenario decision 
points based on providing examples of how experts responded to other scenarios. The 
hypothesis was that providing examples of the way experts responded to similar types of 
scenarios would persuade the experimental group to learn and make decisions like the 
experts, considering and weighing options/actions they may not have in the past. 
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Figure 6. Experiment / Control Group Experiment Matrix 
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At selected decision points (one per scenario), the experimental group completed 
a decision point questionnaire regarding decisions made (see Appendix G). These 
decision points were selected because they provided the opportunity for researchers to 
capture the effects of expert feedback and the SA / RPD model brief on groups’ decision 
making in real time, during the scenario. On a scale of 1-5, with 1 being low and 5 high, 
the experimental group rated the following: The decision difficulty and level of 
confidence regarding the selected decision point. The role that experience (mental 
models), rehearsal (simulation), and pattern matching/cue recognition played in decision 
making and problem solving. Additionally, the three levels of situation awareness, 
perception (size-up), comprehension (making sense) and projection (predicting what will 
happen next) were also rated. The control group did not complete this questionnaire since 
they did not receive the independent variables treatment. 

E. EXPERIMENTAL / CONTROL GROUP SCORING 

First responders often multi-task at emergency incidents, making several 
decisions simultaneously or in consecutive fashion, therefore, when prioritizing from a 
list of potential actions, the individual’s top three responses that agreed with the expert 
response key regardless of order/ranking were scored as correct. For example, an 
individual might select the same three responses as the expert group but rank them in a 
different top order. The response will be considered correct and they will receive full 
credit. However, if the individual’s response matches only two of the three answer key 
response, they received partial credit for that decision point. Table 4 was taken from the 
earthquake scenario decision point # 4.6. At this decision point, the experimental and 
control group members were read the following passage and asked to prioritize a list of 
six possible actions. When compared against the expert response key, both members 
received partial credit for their responses. 
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Table 5. Sample Earthquake Scenario Passage 


Scenario: Earthquake 4 

Upon arrival you observe someone lying face down in front of the station with two men running from 
the location, opposite your approach. 

• The streets in and around the station are crowded with people either requesting help or seeking 
information about the initial earthquake. 

• Incident size-up reveals gasoline leaking from several fuel pumps with fuel pooling in and 
around the station and street. 

• Gasoline has also been discovered coming up through the ground in the rear of the station. This 
fuel is flowing down the street adjacent to the curb line and smoke is venting under pressure 
from the front cornice of the station. 


Scenario Decision Point # 4.6: 

• Prioritize your decision-making. What problems do you address first, second, third...? Explain 
your first three decisions. 

_Fuel coming from rear of station and flowing down the street 

_Person lying face down 

_Smoke from cornice 

_Fuel pooling in front of station 

_People requesting help/information 

_Lack of resources (foam/EMS) 


Expert group's prioritization response key 


Ranking 

Prioritize Decision Making 

1 

Person lying face down 

2 

Fuel coming from rear of station and flowing down the street 

3 

Smoke from cornice 


Experimental group member 


Ranking 

Prioritize Decision Making 

1 

Person lying face down 

2 

Lack of resources (foam/EMS) 

3 

Smoke from cornice 


Control group member 


Ranking 

Prioritize Decision Making 

1 

Lack of resources (foam/EMS) 

2 

Smoke from cornice 

3 

Person lying face down 
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Expert Feedback 

Person lying face down is a known life hazard (visual confirmation) and 
addressing life is a # 1 priority before property. Unit can make quick assessment and 
determine viability of victim removal. The person lying down may possibly be victim of 
robbery and requiring immediate help. 

Fuel coming from rear of station and flowing down the street needs to be 
controlled. If flow cannot be stopped, units must make an attempt to contain or divert 
flow from reaching an ignition source. Leaking fuel is largest threat to life/property, if 
ignited it will intensify an already dynamic and developing situation. 

Extinguishing the fire eliminates a major problem. For example, smoke from the 
fire has potential to expose hospital and apartment building. Investigating the source of 
smoke will reveal fire intensity, determining if a quick stop of fire can be accomplished 
or whether operating personnel should be moved to a safer area 


The earthquake scenario had eight-scenario decision points that added up to 100 
percent. Therefore, each decision point was worth between ten and fifteen points. The 
researcher detennined decision point weight by considering the difficulty of the response, 
impact on the scenario outcome, and number of decision point questions. Scenario 
decision point # 4.6 was worth fifteen percent of total earthquake scenario score; 
therefore both individuals received credit for ten percent because they were correct in two 
of the three responses. 

F. SUMMARY 

Upon completion of the four scenarios, the experimental and control groups 
provided feedback by completing the post-experiment questionnaire (see Appendix H). 
The goal of this experiment was to detennine if the decision-making and problem¬ 
solving skills of newly promoted fire officers, when operating in an asymmetric threat 
environment and without experience, could be enhanced through specific training 

designed to develop those skills. If it can be enhanced through training, then the analysis 
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of scenario decision making amongst the three groups, comparing the experimental and 
control group’s scenario responses with those of the expert groups should indicate a 
significant difference. While this experiment provided greater insight into the hypothesis 
on improving first responder decision making in an asymmetric enviromnent, Chapter VI 
analyzes group response data while making recommendations based on experiment 
findings. 
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VI. ANALYSIS AND RECOMMENDATIONS 


To determine the impact of the independent variables (expert feedback and SA / 
RPD briefing), a one-way ANOVA was used to compare the earthquake scenario scores 
of the experimental and control groups (see Appendix I). Contrary to its namesake, an 
ANOVA is concerned with differences between mean scores and not between variances. 
A one-way ANOVA looks to see if there is a variation between group means. 186 When 
the difference is larger than would normally be expected, you have a significant 
difference. 187 The ANOVA “F” value of 14.087 yielded a “p” value of .001(the 
significance is greater than .05). Therefore, we can say with confidence that there was a 
significant difference between the experimental and control groups’ mean scores for the 
final asymmetrical scenario (earthquake), or the two groups are statistically different 
from each other (see Appendix J). 

Furthermore, since the distinction between the two groups throughout the course 
of the four scenario readings was the experimental group’s receipt of independent 
variables SA / RPD briefing and expert feedback, one concludes that the inclusion of the 
independent variables helped to achieve the dependent variable, namely, improving first- 
responder decision making in an asymmetrical threat environment. 

A. RECOMMENDATION 

Experiment findings indicated that novice decision making was influenced by 
external factors. When comparing the control and experimental groups’ scores on the 
earthquake scenario, novices in the experimental groups were beginning to respond like 
experts. Expert feedback that was provided during practice scenarios was paying 
dividends, an exchange of knowledge had taken place; novice and expert were 
considering the same cues in their decision making, coming to similar conclusions. 

186 Will G Hopkins, “T-Test and One- Way ANOVA,” A New View of Statistics, 2000. 
http://www.sportsci.org/resource/stats/ttest.html . 

187 Susan Archambault, “One-way ANOVA,” Psych 205 Review Site, Psychology Department, 
Wellesley College. 2002. http://www.welleslev.edu/Psvchologv/Psych205/anova.html . 


89 



Several questions surface regarding the thesis hypothesis and experiment finding. 
First and foremost, “How can first responder agencies use experiment findings to 
improve personnel decision making?” In other words, how can they get their novices to 
learn like experts?” There is no need to re-invent the training wheel here. The training 
level of application, however, may need to be elevated. For example, whereas classroom 
instruction often involves the more traditional instructor-led discussions, experience and 
naturalistic decision-making expertise may be best conveyed to the novice via scenario- 
based exercises that provide expert feedback regarding decisions made. 

While designers of scenario-based exercises should strive to focus on domain- 
specific situations that pair expert and novice decision makers together in unfamiliar 
circumstances and have them problem solve; this is not always a requisite of an effective 
scenario. For example, the premise that drove the earthquake scenario was 
inconsequential. It could have easily been a hurricane- or flood-driven scenario and still 
have been effective. Although decision-making situations should be domain-specific 
whenever possible, scenarios merely provide a reason for the decision makers to be there, 
and they should not always be designed with the sole purpose of testing subject matter 
experts. In the case of the earthquake scenario, observing the exchange of information 
between experts and novices — with the intent of developing better decision makers by 
having experts explain the why behind decisions made — was a primary objective. 

For example, scenario-based exercises need to emphasize not only what experts 
consider relevant, but also why. In other words, show the math. If the experts provide the 
answer without explanation, will the learning be short-term? When developing scenario- 
based exercises, designers must be certain that experts expand upon their response by 
describing in detail the cues, patterns, and experiences (mental models) they considered 
when making connections. Did they mentally puzzle out a COA before coming to a 
decision? What was considered in their SA analysis? Otherwise, minus expert guidance, 
the novice gains no advantage when confronting an unfamiliar situation. The scenario 
should drive the exercise, not the decision making. Because, in reality, there may be no 
real experts. 
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So, if there are no or few bona fide experts when situations are novel, how does 
the expert get by? The complexities that constitute an asymmetrical threat environment 
often turn experts into novices for that particular situation, and yet experts can make 
sound decisions. This statement was corroborated when experts, who rated their natural 
disaster knowledge/experience levels barely above average, puzzled out the unfamiliar. 
That capacity — solving a novel situation when information is limited — is what experts 
need to convey to novices. There are several schemes for designing this higher first- 
responder decision-making training curriculums. 

As a fire officer advances through the various promotional ranks, he is often 
required to attend officer training before moving to a subsequent level. One objective of 
this orientation is to familiarize officers with the decision-making and problem-solving 
responsibilities of the new rank. For larger departments, promotional courses may be 
conducted simultaneously, with newly promoted lieutenants, captains, and chiefs each 
attending individual orientation training courses. Experienced fire officers are leaving 
familiar roles regarding decision-making responsibilities, and preparing to assume new 
roles with novel decision-making opportunities. How can departments harness this expert 
knowledge before it leaves the rank? 

Scenario-based simulation exercises, bringing officers of different ranks together, 
may be one answer. This would involve having senior incumbent members preparing to 
leave their current rank work through scenario-based exercises with less experienced 
officers filling the created vacancy. This process is cyclical because, as the “tricks of the 
trade” are conveyed between incoming and outgoing, the officer is both beneficiary and 
benefactor of vital decision-making information. Imagine a group of captains preparing 
for their new rank by working alongside a group of senior chiefs managing a large-scale 
hazardous materials incident via a three-hour scenario-based simulation, and then have 
those same captains school a group of lieutenants regarding the company commander 
responsibilities inherited when assuming the captains’ rank. 

Decision making in today’s asymmetrical environment is as much a concern for 
the newly promoted first-line supervisor responding as a boss to an incident with terrorist 

implications as it is for a three-star chief with over twenty-five years of service assuming 
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command of an escalating incident with no known cause or conclusion. Maybe there are 
no “real technical experts” for asymmetrical incidents just yet, but rather individuals with 
good SA / decision-making skills. Individuals with the ability to make sense of their 
surroundings, using what little cues and patterns the environment offers. Individuals that 
can route decisions through mental rehearsals of plausible COA, project future events, 
and provide expert feedback on decisions made. Sounds insurmountable? Well it is not. 


While participating as an expert in the subway explosion scenario, a well- 
respected FDNY battalion chief with more than thirty years of fire department sendee 
needed to make a decision regarding the stopping of trains while operating at a 
suspected terrorist attack upon a subway platform. Having no mental models with 
subway incidents, he created an analogy with high-rise building fires, with which he had 
a fair amount of firefighting experience. Sounds far-fetched? Not really. The issue with 
the subway incident involved the movement of air; trains provide 90 percen t of subway 
ventilation. Train movement could be beneficial and it could also be catastrophic. Are 
there reports of additional attacks? Are people on the platform? If a chemical agent is 
involved, has it been identified? The chief needed more information. 

Before firefighters provide ventilation at a high-rise building fire, they often go to 
a floor above the fire and open windows, trying to re-create the same conditions units 
will experience on the fire floor should they decide to ventilate the fire floor. The 
firefighter is determining the benefits of ventilation. If a strong wind blows into the 
building, ventilation right now is not a good idea. The fire fighter required additional 
information, not unlike the chief operating at the subway incident. 


This example epitomizes how experts use domain-specific experience from a 
different scenario to create JIT mental models while employing mental simulation to 
puzzle out a COA. The chief was no expert regarding subway incidents, yet he acted 
expertly. This chief problem solved by comparing the workings of a subway system 
encompassing a myriad of underground tunnels and a high-rise building that scraps the 
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sky. By creating this analogy, he decided that the effects of a train moving air, much like 
the wind blowing into the building, was not beneficial to the operation. Imagine the 
benefits of getting a novice to think at this level. 

The New York City Fire Department, along with many other first-responder 
agencies is comprised of individuals with these very talents. They are the experts. How 
long they remain as members of their respective departments in another matter. While 
they still are, we need to learn from them. 


A well-known golf instructor commenting on the art of teaching the motions of his 
golf swing to a novice, comments there is little an expert can do regarding teaching a 
swing. The learning needs to be experiential and feedback based. This golf analogy goes 
for SET... “It’s about learning, not about golf. ” 188 


188 Ayers, et at, “Managing Risk through Scenario Based Training.” 
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APPENDIX A 


Pre-experiment questionnaire mean scores 

In addition to asking the three groups to provide their years of service with the 
fire department and current rank, the pre-experiment questionnaire also asked the groups 
to rate their experience/knowledge level in five emergency response categories. On a 
scale of 1-5, with 5 indicating a high level of experience, the experimental group rated 
their natural disasters experience/knowledge level with a mean rating of 1.8, indicating 
the least experience regarding the five categories. The control groups’ ratings for natural 
disasters were 2.7, while the experts’ was a 3. 


Groups 

Experts: (10) Chiefs (5) Captains 

Experimental: (14) Lieutenants 

Control: (5) Captains (10) Lieutenants 

Expert 

(14) 

Experimental 

(14) 

Control 

(15) 

Years of service with the Fire Department 

25 

9 

12 

Years of service in current rank 

5 

0 

1 


On a scale of 1-5, with 1 indicating little experience 
and 5 indicating extensive experience, rate your lire 
department knowledge/experience in the following 
response categories 

Expert 

Experimental 

Control 

Knowledge/experience with high-rise building 
firefighting 

3 

2.4 

2.3 

Knowledge/experience with subway related incidents, 
(i.e.) fires/emergencies 

4 

2.8 

1.3 

Knowledge/experience with CBRNE related incidents 

3 

2 

2.5 

Knowledge/experience with hazardous material 
incidents 

3 

2.1 

2.9 

Knowledge/experience with natural disasters 

3 

1.8 

2.7 
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APPENDIX B 


Pre Exercise SA / RPD Group Briefing 


Situational Awareness (SA): Simply stated, SA is the art of being aware of your surroundings via SA 
perception, comprehension, and projection levels. 

Level 1 SA Perception: 

(initial size-up) 

Knowledge of surroundings, recognizing familiarities along with 
inconsistencies. 

Level 2 SA Comprehension: 

(making sense of surroundings) 

Understanding/comprehending what you perceive along with its 
significance to incident goals and objectives 

Level 3 SA Projection: 

(predicting what will happen next 
and take appropriate actions) 

Projection of future events (what will you do with the information), 
governed by the synthesis of perception and comprehension. 

Recognition-Primed Decision Model (RPD): The recognition-primed decision model describes the way a 
decision-maker is able to recognize a situation and make a decision by comparing current events to stored 
experiences. 

Cue recognition: 

(Color of smoke; volume of fire) 

Identification and evaluation of incident cues. Cues result in patterns; 
patterns are compared to stored experiences, (mental models) 

Pattern matching: 

(water normally cools & extinguishes 
fire) 

“Pattern matching refers to our ability to detect typicality and notice 
events that did not happen and other anomalies that violate the 

pattern.” 189 

Mental models: 

(private dwelling attic fires often 
originate in the basement) 

Mental models explain the way things work! They are our stored 
experiences that help us describe, explain, and predict situations. 190 

Mental simulations: 

(mentally rehearing the steps of a 
roof rope rescue moments before 
initiating) 

We evaluate a course of action by consciously imagining what would 
happen if we carried it out. 191 


1^9 Gary Klein, Sources of Power, 149. 

190 Kevin Bums, ‘‘Mental Models and Normal Errors,” (eds) Berndt Brehmer, Raanan Lipshitz, Henry 
Montgomery, How Professionals Make Decisions (Mahwah, New Jersey: Lawrence Erlbaum Associates, 
2005): 17. 

191 Klein, Power of Intuition, 26. 
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• Cues let us recognize patterns 

• Patterns activate action scripts 

• Action scripts are assessed through mental simulation 

• Mental simulation is driven by mental models ' 92 



Klein, Power of Intuition, 28. 
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APPENDIX C 


Renegade tanker on bridge scenario 

Scenario: Renegade tanker on bridge 1 

As the first engine company (pumper) officer to arrive at the scene of a 911-generated 
response for a truck fire on a suspension bridge during evening rush hour, you receive the 
following information from police stationed at the foot of the bridge: 

• A tanker with a capacity of 5,000 gallons disregarded police instructions and 
attempted to use the bridge’s lower level that is restricted to passenger traffic 
only. A police vehicle immediately pursued the tanker. 

• While attempting to stop the tanker before reaching mid-span, the police car 
crashed into the center divider and flipped over, while causing the tanker to 
sideswipe a passenger bus traveling in the same direction. 

• The bus swerved before coming to an abrupt stop, as the tanker crashed into the 
side rail with the driver of the tanker fleeing the scene. 

• Bridge police are diverting traffic on the bridge while attempting to contact the 
police officer involved in the chase. 

Upon receipt of the preceding infonnation, you establish command at the bridge. 


Scenario Decision Point #1.1: 

• In regards to the preceding scenario, prioritize the following list of actions. 
Explain your first three actions 

_ID product leaking from tanker 

_Life hazards to occupants of bus and police car 

_Whereabouts of driver leaving scene 

_Protecting first responders /risk vs. reward 

_Communication w/units and dispatcher/ request additional alarms/resources 

_Security of scene/ isolation of area against additional tankers/explosions (terrorism) 

_Prevent contamination 

_Notify additional agencies 
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Expert group’s response key 


Ranking 

Actions 

1 

Comm, w/units and dispatcher/request additional alarms/resources 

2 

Protecting 1 st responders / risk vs. reward 

3 

Life hazard to occupants of bus and police car 


Expert Feedback: 

Communication with units and the fire dispatcher was a common thread among 
expert decision makers. The experts overwhelming believed that incoming units needed 
to be alerted regarding the magnitude of the event. Such an alert would be helpful to 
responding units regarding possible staging locations while ensuring units don the proper 
protection equipment (PPE). Operating units also needed to request, via the dispatcher, 
the response of specialized units and resources. For example, a hazardous material unit 
may be needed to confirm the identification of the ta nk er’s contents and mitigate the leak; 
a foam unit may be required to extinguish burning fuel, and EMS to address medical 
needs. 

At an incident of this type, the incident commanders must consider the “risk vs. 
reward” factor. Will rescues endanger first responder personnel? Protecting first 
responders, preventing them from becoming part of the problem is the most important 
consideration in the size-up. If first responders are not safe, they cannot help those they 
are entrusted to protect. For example, first responders may be conducting operations 
amongst an unidentified leaking product; therefore, they must consider possible exposure 
to and contamination from a hazardous material and take appropriate protective actions. 

The life hazards to the police vehicle and occupants of the bus is paramount. An 
initial size-up of the two vehicles will generate a resources list. For example, ambulatory 
victims will be immediately removed while non-ambulatory and those pinned/trapped 
will require special resources. The number of injured bus passengers and status of police 
officer will determine the EMS’s commitment of resources. 


100 




Scenario: Renegade tanker on bridge 


1 


Approaching the incident, you observe the following conditions: 

• The engine compartment/cab section of the tanker is involved in fire while 
exposing the passenger bus 

• A steady flow of liquid coming from the rear compartment of the tanker 

• An occupied passenger bus stopped about 25 feet beyond the tanker 

• A police vehicle lying on its side about 25 feet in front of the bus 

• The product has been identified as # 2 fuel oil 


Because bridge traffic is now at a standstill in both directions, additional units will be 
delayed in gaining access to the immediate area. 


Scenario Decision Point # 1.2: 

• What will you address first? Prioritize and explain the following decision¬ 
making options (1, 2, 3, 4): 

_Police vehicle lying on its side 

Fuel tanker involved in fire 


Liquid coming from the rear compartment of the tanker 
Occupied passenger bus 



Expert group’s response key 


Ranking 

Decision making options 

1 

Fuel tanker involved in fire 

2 

Occupied passenger bus 

3 

Police vehicle lying on side 
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Expert Feedback 

Addressing the burning fuel tanker was an overwhelming concern of most 
experts. Extinguishing the fire solves numerous problems and has the greatest impact 
regarding the protection of life and property. If total extinguishment of the fire cannot be 
attained due to the contents burning, then at least containment can be achieved. If the fire 
is not addressed, the incident can expand. Additionally, because the fire exposes the 
passenger bus, containment would assist the rescue effort while reducing panic among 
bus passengers. 

The bus is occupied, and therefore a top priority. It is severely exposed to fire, 
with a good possibility of containing injured passengers. The police vehicle lying on its 
side is the furthest from the incident and may possibly require a technical rescue 
(extraction), and has the least impact upon the incident. Although the liquid leaking from 
rear of tanker is a concern, most first responders are not trained to the level required to 
mitigate the leak that is both time consuming and requiring special equipment. 
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APPENDIX D 


Explosion on subway platform scenario 


Scenario: Explosion on subway platform 


2 


You are a newly promoted lieutenant working a day tour in a mid-town engine company 
(pumper). 


• At 5:00 pm, your company responds to a 911-generated dispatch, reporting a loud 
noise on a subway platform within your first-alarm district. 

• The response to this alarm consists of (2) two engine companies and (1) one 
ladder company; The Battalion Chief normally assigned is operating at another 
incident and is unavailable. 

• Your company normally arrives second at this location, however, while in route 
you receive notification that you are now arriving first and that other loud noises 
have been reported at subway platforms serviced by the same subway line. 


Scenario Decision Point # 2.1: 


• Does the receipt of additional information influence your decision-making? 
Yes/No. Explain. 


Group 

Yes / No 

Percentage 

Experts 

Yes 

100 % 


Expert Feedback 

The receipt of additional infonnation influenced experts’ decision making in the 
following ways: First, an isolated incident on a subway platform could be the result of an 
electrical fire, derailed train car, or possibly a trash can fire, just to name a few. While 
these events are not everyday occurrences, they do happen from time to time and 
typically are isolated events. Multiple incidents, however, on the same platform or at 
more than one location and you begin to recognize a pattern. Given the location (mass 
transit platfonn) and the history of past attacks on similar locations (Madrid and Tokyo), 
first responders need to think terrorism or premeditated planning. For example, several 
loud noises equate to explosions and explosions equate to terrorism. Additional reports of 
explosions confirm an ongoing event and verify a consistency in information received 
while lending credibility to earlier reports whether terrorist related or not. 


103 






Second, your unit’s responsibilities change because of additional information. 
Because you will now arrive first on the scene, you will assume the role of incident 
commander (until relieved by a higher rank) and be responsible for transmitting a 
preliminary report regarding initial size up. Depending on rank structure and arrival order 
of responding units, your initial role/responsibilities may be more strategically (directing 
operations) and less tactical. 

Scenario Decision Point # 2.2: 

• Prioritize the following concerns. Explain your first three concerns. 

_Evaluate status of victims and evacuation in progress 

_Establish communication with the transit authority for additional infonnation 

_Recognize Incident Commander responsibilities 

_Conduct environmental size up (smoke, fire, readings) 

_Adequate resources on scene 


Expert group’s response key 


Ranking 

Concerns 

1 

Recognize Incident Commander responsibilities 

2 

Conduct environmental size up (smoke, fire, readings) 

3 

Evaluate status of victims and evacuation in progress 


Expert Feedback 

The first officer assuming command at an incident of this type needs to recognize 
that early establishment of command and control is imperative for the safety of operating 
personnel. Standard firefighting operating procedures, often predicated on an aggressive 
attack of the fire to protect life and property may not be applicable here. Instead, more 
time than usual is required in initial size-up. While your unit’s initial duties may have 
included the stretching of hose lines or identification of the hazard, you are now 
responsible for the overall management of the incident. 

As the incident commander, you need to conduct a size up, provide direction 
(leadership), and prioritize actions (what needs to be done first). Two primary concerns 
include unit deployment and the transmission of preliminary reports to notifying higher 
command of a developing situation, while alerting incoming units regarding the 
magnitude of the event. 
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An environmental size-up regarding smoke, fire and monitoring of the immediate 
vicinity for hazardous material release will determine future actions. At an incident of 
this type, units cannot continue operations without additional information. There are too 
many unknowns. For example, reports of loud noises on a subway platfonn are not 
unusual; however, they do need to be investigated. A visual size-up of the subway air 
vents at street level may indicate conditions down on the subway platfonn. Where there 
is smoke there is fire! Units need to create patterns from cues, reports received, and 
expectations developed from past events. 

Life is a #1 priority. At an incident of this type, the incident commander must 
evaluate the status of victims and evacuation in progress. Signs and symptoms of 
evacuating passengers provide information regarding the type of event. What symptoms 
do victims normally exhibit when evacuating a fire related environment? Are the 
passengers exhibiting signs/symptoms that would normally be expected, or are they 
displaying signs and symptoms that are atypical? The evacuation status also provides 
valuable information regarding first responder commitment. For example, our mental 
models / expectations regarding rush hour at a subway platfonn tells us to expect a large 
number of passengers using the subway station. Therefore, while responding, we 
anticipate or expect to find a mass exodus up the subway platform stairs to the street 
above. 


105 



Scenario: Explosion on subway platform 


2 


Upon arrival at the street-level entrance to the subway platform, you observe several 
people with minor injuries complaining of breathing difficulties; there is a light to 
medium smoke condition coming from the subway vents. 


• On-scene transportation personnel inform you that there have been a total of three 
loud noises, “The first noise occurred immediately after the train left the station 
and shortly after two subsequent noises were heard—first one then the other.” 

• A surveillance camera located at the far end of the station reveals two people 
overcome on the platform as several small fires burn. Numerous backpacks and 
bags are scattered throughout. 

• Presently there is no one exiting the subway via the street entrance where you are 
located. It is now 5:15 pm and the remaining assigned units are arriving. 


Scenario Decision Point # 2.3: 


• Are you committing personnel to the subway platform? Yes / No. Explain. 


Group 

Yes 

No 

Experts 

60% 

40% 


Expert Feedback 

Sixty percent of experts committed personnel to the platform. The common thread 
amongst the experts was the now confirmed life hazard. The signs/symptoms that 
evacuating passengers displayed were also instrumental in the IC decision to commit 
units to the platform. For example, people complaining of breathing difficulties while 
serious, is a common fire related symptom (mental models from experience). The smoke 
could be the result of a malicious trash can fire, with the noise being the sound of a 
heated aerosol container exploding. A majority of the experts believed that a limited 
commitment of resources outfitted with full protective equipment to rescue victims, 
extinguish fires and conduct a reconnaissance of the immediate area was a justifiable risk 
considering the rewards. 

Conversely, forty percent of the experts decided against committing personnel to 
the platfonn without additional infonnation. The common thread amongst this group of 
experts was that further investigation of the scene was required, identifying hazards 
before committing personnel. This incident could involve an improvised explosive device 
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with possible chemical or nerve gas components. Experts made this observation by 
considering the history of past incidents (sarin gas attack on Tokyo subway system). The 
limited evacuation of subway platform does not match patterns and expectations for rush- 
hour time of day and reinforces reluctances to commit personnel to an unknown 
environment. Additionally, the use of first responder radios could detonate secondary 
explosive devices designed to hann first responders operating at the scene (Madrid train 
bombings). 

Scenario Decision Point # 2.4: 


• At this point, are you stopping train movement? Yes / No. Explain. 


Group 

Yes 


Experts 

90% 



Expert Feedback 

Ninety percent of the experts agreed that train movement should be stopped. The 
moving trains provide subway tunnel ventilation; therefore stopping trains reduces air 
movement and contamination of adjoining stations. Additional reasons for stopping trains 
included the following: Isolating trains at previous station prevents trains from entering 
the station and exposing passengers to a possible secondary explosive device; passengers 
may have entered the track area because station was uninhabitable or exited stranded 
trains in tunnel between stations; the incident commander may need to commit units to 
the track area and confirmation of power removal takes time. 
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Scenario: Explosion on subway platform 


2 


From an adjacent subway entrance that serves the same platform but not within your line 
of sight, units report that they are operating at street level assisting in a mass evacuation 
of subway passengers. 

• There are some minor injuries and complaints of difficulty with breathing, 
however for the most part passengers appear to be ambulatory. 

• Evacuating passengers inform assisting units that there was a loud noise just as 
the train pulled out of the station and then everything went black. 


Scenario Decision Point # 2.5: 

• Does the receipt of this information change your current decision-making 
(committing personnel to platform / train movement)? Yes / No. Explain. 


Group 

No 


Experts 

70% 



Expert Feedback 

Seventy percent of the experts stayed with initial decision-making. Reasons for 
staying with initial plans include that the new information (regardless of their prior 
decisions to commit or not commit personnel to platform or stop trains) does not 
significantly influence their strategies regarding decision to commit or not commit units 
to platform and stopping train movement. For those who decided to commit personnel to 
platform the new infonnation reinforces the initial objectives, namely that victims are 
endangered and need to be rescued. Also, the new information is consistent with their 
original assessment and expectations of explosive situation and decision to commit has 
already been initiated. For the experts deciding against the commitment of personnel, the 
situation still remained undefined and a good assessment needs to be made before 
committing personnel. 

Scenario Decision Point # 2.5: 

• Prioritize the following actions. Explain your first three actions. 

_Ensure adequate resources on hand 

_Transmit information to dispatch (progress report) 

_Guard against secondary devices/terrorism 

_Need additional infonnation from TA 

_Consider gross/mass decontamination of passengers 

_Limited search of subway w/PPE ensuring responder safety 
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Monitor immediate environment 


Expert group’s response key 


Ranking 

Actions 

1 

Limited search of subway w/PPE ensuring responder safety 

2 

Guard against secondary devices/terrorism 

3 

Transmit information to dispatch (progress report) 


Expert Feedback 

Because life is a # 1 priority, a risk vs. reward evaluation warrants a limited 
search of subway platform. Adhering to PPE recommendations allows first responders to 
achieve goals of rescue and reconnaissance while at the same time retaining a level of 
safety. 

Other than a visual inspection, initial units do not have the capability to detect the 
presence of an explosive device. With this in mind, units need to guard against secondary 
devices designed to kill first responders, while recognizing their limits when operating at 
the scene of initial explosion. If units consider this incident to be terrorist related, they 
must recognize that the use of secondary explosives devices is a pattern of past terrorist 
attacks (Madrid) and guard against complacency and request additional resources. 

Special first responder resources are required at this incident. Their use should be 
recognized and requested as early as possible in the incident. For example, victims on the 
platform and evacuating passengers requiring medical attention will require a greater 
commitment of EMS resources besides those normally assigned on the initial response. 
The immediate environment must be monitored by special equipment not normally 
assigned to first responder units. Lastly, higher levels of command need a size up-this 
incident is part of larger problem 
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Scenario: Explosion on subway platform 


2 


A few minutes later, several evacuated passengers are exhibiting signs of weakness and 
confusion and complaining of headaches, dizziness, and nausea. Three victims become 
incapacitated and require immediate medical attention. 


Scenario Decision Point # 2.6: 

• On a scale of 1-5, with 1 being very low and 5 being very high, how confident 
are you that this incident is terrorist related? Explain. 


Group 

Scored a 4 or higher 

Expert 

70% 


Expert Feedback 

Seventy percent of the experts scored a 4 or greater regarding their level of 
confidence that this incident is terrorist related. Location, multiple explosions, and 
medical symptoms all point to terrorism. Similarities can be drawn between this attack 
and past attack on comparable locations (Madrid train bombing). For example, terrorist 
groups are known for attacks on critical infrastructure (transit) along with the time of day 
(rush-hour) and the large number of passengers. Reports from evacuating passengers that 
an explosion occurred immediately after the train left the station coupled with delayed 
physical symptoms and victim incapacitation are indicative of a chemical type attack 
reinforcing beliefs that this incident is terrorism related. 

Scenario Decision Point # 2.7: 

• If this incident is terrorist related, what type of attack might the scenario 
information indicate? 


Type of attack 

Chemical 

Explosive 

Dirty bomb 

Incendiary 






Experts 

7 (50%) 

4 

2 

1 


Scenario Decision Point # 2.8: 

• Prioritize the following decision-making actions. Explain your first three 
actions: 

_Safety full PPE 

_Request additional resources for decontamination 

_ Make necessary notifications 
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Ensure passenger evacuation continues 
Isolate and confine access to area 
Extinguish fires burning 
Triage of victims 

Re-examine monitoring of environment 
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Expert group’s response key 


Ranking 

Actions 

1 

Safety full PPE 

2 

Make necessary notifications 

3 

Isolate and confine access to area 


Expert Feedback: 

Experts agreed that in addition to ensuring that all members adhere to PPE 
recommendation and notifications regarding incident dynamics continue to be 
transmitted, the immediate area must be isolated. The latest information regarding several 
evacuated passengers exhibiting signs of weakness, confusion, and complaining of 
headaches, dizziness, and nausea, along with victims becoming incapacitated and 
requiring immediate medical attention necessitates that the area be sealed. Intrusion from 
the exterior and contaimnent of those contaminated / injured until further evaluation in 
the form of decontamination and triage can be administered is paramount. If isolation 
measures are not taken, the incident has the capability of expending beyond immediate 
area. 
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APPENDIX E 


Radiological response scenario 


Scenario: Radiological response 

3 

A highly publicized and controversial meeting scheduled for the United Nations builc 

ling 


has resulting in several credible threats from dissent groups vowing to take unspecified 
action if the meetings are not canceled. 

• Due to the seriousness of these threats, along with the importance of these 
meetings taking place, all affected first responders have increased their awareness 
levels. In addition to their normal security precautions, the police are spot 
searching vans/trucks entering Manhattan, while fire department units have been 
reminded of the importance to activate and monitor all CBRNE-related equipment 
while out of quarters. 

• Although all threats are taken seriously, precautions of this nature are not unusual 
for units that work in this area. More times than not, these threats are 
unsubstantiated, the awareness level is reduced, and units go back to business as 
usual. 

• Throughout the first two days of the United Nations meetings, no unusual 
occurrences have been reported; however, your unit’s false-alarm rate during this 
two-day period has increased by more than 100 percent. 


Scenario Decision Point #3.1: 

• Given the following information, how do you prepare for the your upcoming 
tour of duty on the final day of the United Nation’s meeting? Prioritize the 
following actions. Explain your first three actions. 

_Check pass along book regarding responses from previous tours 

_Remind members to guard against complacency due to the unusually high false 

alarm rate 

_Ensure all equipment is operational and members are knowledgeable with 

equipment use 

_Contact surrounding companies for additional information 

_Contact Police Department for update 

_Prepare Drill on CBRNE type attack 

_Conduct briefing stressing the type of threats units could encounter 
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Expert group’s response key 


Ranking 

Actions 

1 

Ensure all equipment is operational and members are knowledgeable with 
equipment use 

2 

Check pass along book regarding responses from previous tours 

3 

Remind members to guard against complacency due to the unusually high false 
alarm rate 


Expert Feedback 

The experts agreed that ensuring all equipment is operational and members are 
knowledgeable with equipment use was a top action. Understanding that predicting what 
might happen on any given tour is impossible and that other than signs / symptoms, the 
only way to detect the presence of radiation is through the monitoring of the 
environment, working equipment and knowledge of its use is imperative. 

The placing of equipment in or out of service is just one of the many entries that 
may be found in a unit’s pass along book. Experts believed that one of the first steps in 
preparation for the upcoming tour was to check unit’s pass along book entries recorded 
since their last tour of duty. Entries will capture any unusual events that have occurred on 
prior tours creating patterns regarding specific types of response. For example, a unit may 
record notifying the next level of command regarding the unusually high false alarm 
response rate to a specific location. 

Pattern of high false alarm rate with no explanation reinforces the need to remind 
members to guard against complacency due to the unusually high false alarm rate. For 
example, terrorist may be monitoring response patterns to certain locations, waiting for 
apathy to sets in, resulting in firefighter complacency followed by terrorist strike 
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Scenario: Radiological response 


3 


At 11:00 am on the third and final day of the meetings, while responding first due to a 
street-box alann that requires you to pass within one block of the United Nations 
building, your unit receives a reading of (1 mr/hr) on their radiological detector. This is 
the first radiological reading units have received since the meetings commenced. The 
alarm location you are responding to has been one of your recent chronic false alarms 
boxes. 


Scenario Decision Point # 3.2: 

• Do you stop to investigate or proceed to original response location? Stop / 
Proceed. Explain. 


Groups 

Proceed 

Stop 

Experts 

60% 

40% 


Expert Feedback 

Sixty percent of the experts believed that unit must continue to original response 
location. No known life hazards along with radiological readings too low or not unusually 
high (possibly background readings) were among reasons for proceeding to original 
location and not stopping to investigate. Units did notified the dispatcher regarding 
readings (location/rate) while passing along information to units responding to original 
alarm and requested the response of nearest available unit or hazardous material unit with 
sophisticated equipment to verify readings. 

Forty percent of the experts believed readings were high enough to warrant 
investigation for verification and that some type of radiological event has taken place. A 
reading of 1 mr/hr on apparatus may indicate higher reading closer to source and units 
may be approaching an area with higher readings. Unit notified dispatcher of readings 
and that they are stopping to investigate while requesting an additional unit to verify 
readings with a second meter. Additional units can continue to original alarm location 
that has been chronic false alarm box. 
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Scenario: Radiological response 


3 


A few moments later your unit is redirected to a reported traffic accident one block north 
of the United Nations. While entering the block of the reported location, your apparatus 
narrowly avoids a head-on collision with a car driving against oncoming traffic. The car 
comes to an abrupt halt with two men exiting and running away from the scene. You 
continue to monitor your readings that have increased and are now holding steady at 
(lOmr/hr). Sizing-up the accident scene you observe the following: 


• A traffic accident involving a van and a small bus transporting about 10 
passengers. Writings on the side and front of the bus are in English and Hebrew 

• The van has crashed into the side of the bus pinning the bus driver to his seat and 
blocking the only exit door on the bus 

• A small fire bums under the van 

• You are now receiving a reading of (5r/hr) on your radiological detector 


Scenario Decision Point # 3.3: 

• Considering the “risk vs. reward” commitment of resources, what will you 
address first? Prioritize the following decision-making options. Explain your 
first three decision-making options. 

_Address life hazard on the bus 

_Transmit size-up and resource request 

_Extinguish the fire 

_Identify hazards and monitor environment 

_Ensure all members wear full PPE 

_Isolate the area due to confirmed radiological threat (hot zone) 


Expert group response key: 


Ranking 

Options 

1 

Ensure all members wear full PPE 

2 

Transmit size-up and resource request 

3 

Address life hazard on the bus /Isolate the area 


Expert Feedback 

Ensuring that all personnel wear full PPE protects firefighters so they can protect 
life. Keeping firefighters from becoming victims themselves is an incident commander’s 
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primary concern. At this incident, the inhalation of contaminants is an immediate threat 
to firefighters. The verified radiological readings (5 r/h) in immediate area of bus and van 
dictate that firefighter don PPE, providing sufficient protection for short-term exposure 
(alpha / beta). 

This incident is beyond the capabilities of a single unit response, necessitating 
that units transmit a size-up alerting incoming units regarding the magnitude of the 
incident while requesting additional resource. A size up alerts higher command to 
escalating incident while providing valuable information to responding units. For 
example, incoming units need to know location of contamination controls zones. A 
resource request gets specialized units and equipment on the scene (Haz-Mat 
unit/decontamination) or at least on the road, additional manpower affords the incident 
commander the ability to rotate personnel. 

Isolation of immediate area is required pending further size up while protecting 
members from additional exposure. Reading of 5 r/h confirms a radiological threat and 
necessitates the evacuation of immediate area and establishment of contamination control 
zones to identify areas requiring special equipment and personal protection levels. Life 
hazard on the bus is a #1 priority. There is a medium risk (exposure) for high reward 
(life). Units can make rescues, evacuating ambulatory passengers first and rotating 
personnel for technical rescues limiting exposure to radiation. 


Scenario: Radiological response 


3 


While sizing-up the incident, your chauffeur informs you that additional units are being 
assigned to this location and that similar incidents are being reported throughout the area. 


Scenario Decision Point # 3.4: 

• Does this new information change your decision making process regarding 
your unit’s commitment? Yes / No. Explain. 


Group 

No 


Experts 

80% 
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Expert Feedback 

Eighty percent of the experts were already treating this as a terrorist related 
incident, taking the necessary precautions and actions to ensure firefighter safety (PPE, 
isolation, notifications). They were committed to incident and recognized immediate 
problems i.e., life, fire, hazardous materials, etc...The additional information confirms 
assumptions regarding terrorism and heightens the likelihood that this is a terrorist related 
event, however it does not change decision making process — objectives regarding the 
incident remains the same , a radiological attack w/victims. 
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APPENDIX F 


Earthquake scenario 


Scenario: Earthquake 


4 


It is January 2, 2008, and you are working a day-tour in Ladder Company 999 located in 
Queens, New York. 

• Ladder 999’s response area is mainly residents consisting of private dwelling 
wood-frame buildings that are two to three stories in height. Although the 
neighborhood is composed of several ethnicities, most of the local residents are 
Hispanic. 

• While in the process of completing your 0900 roll call and familiarizing yourself 
with apparatus tools and equipment, the ground beneath you begins to shake 
causing you to momentarily lose your balance, as breaking glasses/dishes can be 
heard from the kitchen area. 

• The shaking lasts no more than twenty seconds; however, it is immediately 
followed by two additional tremors that are shorter in length and less severe. The 
distinguishable error/failure tone emitting from the house watch computer is 
drowned-out by the high-pitched sound of car and house alarms heard up and 
down the street of the firehouse. 


Decision Point # 4.1: 

• From the information received up to this point, prioritize the following list of 
decision-making actions and explain your first three actions. 

_Monitor media for additional information (television/radio) 

_Establish communications w/dispatcher 

_Prepare for emergency type responses (gas and water mains, downed trees/power 

lines) 

_Assessment of members/firehouse 

_Prepare for verbal alarms received at firehouse 

_Establish communications with immediate supervisor 


Expert group’s response key 


Ranking 

Action 

1 

Assessment of members/firehouse 

2 

Establish communications w/dispatcher 

3 

Establish communications with immediate supervisor 
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Expert Feedback 

The condition of personnel and structural stability of firehouse is the primary 
concern. Firefighters cannot help civilians if they are unable to respond because of 
personal injuries or structural damage to firehouse (utilities, apparatus doors, apparatus 
floor, and apparatus). Understanding that few fire officers are structural engineers, a 
visual inspection of the firehouse building needs to be conducting nonetheless. Among 
the questions that need to be answered include: Can the building withstand a similar 
tremor? Does unit need to evacuate due to structural damage to firehouse? 

By contacting dispatcher, unit establishing a reliable communication link while 
notifying the dispatcher of unit’s availability for response. For example, if units cannot 
open the firehouse apparatus doors, they cannot respond. Units can also determine extent 
of earthquake damage outside of their immediate response area. Is it widespread or 
limited in scope? This is important because unit will know how much support it can 
expect from surrounding units. Communication with dispatcher will also help to appraise 
next actions. 

In addition to establishing communication with dispatcher, units also need to 
contact the next level of command. The chief will be gathering additional information to 
determine safest response matrix, therefore they need to know unit status and availability. 
For example, if a unit is in-service but responding undennanned, the chiefs priorities 
may need to change regarding the protection of life and property, they may not be able to 
perform both. 
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Scenario: Earthquake 


4 


You find out that at approximately 8:58 am, the New York City area experienced an 
earthquake with a reported 5.5 to 6.0-magnitude range. Tremors have been reported for 
up to 100 miles of the epicenter located just north of the city with additional aftershocks 
reported throughout your area. 


• At 9:10am, you are dispatched via department radio to a reported explosion and 
tire at a gas station located within two blocks of the firehouse. 

• Before leaving quarters, you receive information regarding a possible robbery in 
progress at your response location. Additional fire units have been assigned to the 
alarm and the police are in route. 

• While responding you are flagged down by neighbors telling you that their child 
is hurt and needs medical care. Although you are able to understand bits and 
pieces of the conversation, the language barrier prevents you from obtaining any 
additional information regarding the extent and circumstances surrounding the 
child’s injuries. 


Scenario Decision Point # 4.2 

• Does this new information regarding a possible robbery in progress and 
verbal alarm for injured child change your response procedures? Yes / No. 
Explain. 


Expert group’s response key 


Group 

Yes / No 

Percentage 

Experts 

Yes 

100 % 


Expert Feedback 

Experts overwhelming agreed that the child was an immediate life threat and must 
be addressed. Personal contact with civilian (verbal alarm) confirms an incident that may 
require prompt action if life threaten. Prioritizing this incident will influence next 
decision regarding unit remaining on the scene or proceeding to original alarm. If injuries 
are not life threatening, the officer may consider leaving one firefighter with child until 
arrival of EMS. 
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Scenario: Earthquake 

4 

You stop and discover the child’s injuries to be minor and already under the care of El 
In the meantime, the dispatcher has received several calls reporting smoke and a st 
collapse in the same vicinity of the gas station. The police are also reporting widespi 
looting throughout the area. 

MS. 

reet 

*ead 


Scenario Decision Pt # 4.3: 

• Place a check next to the following decision-making actions/options you agree 
with and rank which actions/options are most important. Explain your first 
three actions. 

_Proceed to original alarm but wait for police before investigating 

_Advise chauffeur to slow down and proceed with caution 

_Inform members of additional information regarding reports of a robbery inprogress 

and street conditions 

_Use of heavy caliber streams from a distance 

_Wait for remaining alann assignment before continuing into immediate area 

_Consider hydrant system to be compromised 

_Due to reports of civil disturbance, maintain unit integrity when operating 


Expert group’s actions/options response key 


Ranking 

Actions / Options 

1 

Inform members of additional information regarding reports of a robbery in 
progress and street conditions 

2 

Advise chauffeur to slow down and proceed with caution 

3 

Due to reports of civil disturbance, maintain unit integrity when operating 


Expert Feedback 

Upon arrival at the scene, operating units need to be on same page regarding 
operations. By informing members of additional information regarding reports of a 
robbery in progress and street conditions, members will understand circumstances of 
response before arrival 

Anticipating the infrastructure to be compromised (traffic lights, gas, electric), 
along with the possibility of further street collapse occurring due to weight of apparatus 
and undennined street from water main breaks, necessitates that the officer and chauffer 
communicate regarding response. 
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Do not allow members to become victims of civil disturbance/robbery. Operating 
as a team (no member operates alone) maintains unit integrity, allowing officers to assess 
the “big picture” before committing members to individual tasks 


Scenario: Earthquake 


Because your response route is obstructed, you have to navigate around several displaced 
vehicles and assorted debris. 

• You observe numerous small fires burning in the distance, but no significant 
damage, other than a few downed trees and electrical wires detected within your 
immediate vicinity. 

• Your chauffeur informs you that this gas station is situated in an area comprised 
of two ten-story residential buildings and a two-story hospital. 

• Monitoring the department radio, you are aware that fire/EMS units are 
overwhelmed with medical, gas odors, electrical, water pressure problems, 

_etc.,... types of emergencies._ 


Scenario Decision Point # 4.4: 

• Do you anticipate that the hospital or residential buildings will be an 
immediate concern? Yes / No. 


Group 

Yes / No 

Percentage 

Experts 

Yes 

100 % 


Scenario Decision Point # 4.5: 

• Prioritize the following list of decision-making situations you might encounter. 
(1 being the most important and 9 the least). Explain your first three decision¬ 
making situations. 

_Stuck elevators at both locations 

_Exposure problems from possible falling glass (ten-story buildings) 

_Extension of fire from gas station 

_Possibility of multiple high-pressure gas leaks at location 

_Movement of patients may become an issue (evacuation or shelter in place) 

_Limited emergency -type resources available to assist 

_Smoke (hydrocarbons) from fire may affect hospital and apartment complex 

_No electrical power in area 

_Request for assistance from hospital staff or apartment residents 
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Expert group’s response key 


Ranking 

Decision-making situations 

1 

Extension of fire from gas station 

2 

Possibility of multiple high-pressure gas leaks at location 

3 

Limited emergency type resources available to assist 


Expert Feedback 

Extension of fire from the gas station presents an immediate hazard with greatest 
potential to threaten life/property. Although units are already overwhelmed, by 
addressing the fire, they have the opportunity to impact and influence the situation. 
Because resources are limited and water source questionable, fire must be contained and 
prevented from extending (not necessarily extinguished). The situation will get worse if 
fire extends — do not allow fire extension to add to problem. 

Not unlike the fire, gas leaks, if not controlled, have the potential to significantly 
impact the incident. The availability of multiple ignition sources throughout area (fires, 
downed electrical lines), along with the potential of ignited gas leaks under pressure 
leading to a fast-moving fire, necessitates units consider such actions as building 
evacuation and gas dispersion. 

Due to limited availability of emergency-type resources, the magnitude of 
incident will overwhelm resources. If the incident is widespread, numerous events will 
have taken place, and requests for additional resources will be a priority and prevalent 
throughout city. The strain on personnel to do many things will be great. Units must 
prepare to be self-sufficient while recognizing their limits. 


Scenario: Earthquake 


4 


Upon arrival, you observe someone lying face down in front of the station with two men 
running from the location, opposite your approach. 

• The streets in and around the station are crowded with people either requesting 
help or seeking information about the initial earthquake. 

• Incident size-up reveals gasoline leaking from several fuel pumps with fuel 
pooling in and around the station and street. 

• Gasoline has also been discovered coming up through the ground in the rear of the 
station. This fuel is flowing down the street adjacent to the curb line and smoke is 
venting under pressure from the front comice of the station. 
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Scenario Decision Point # 4.6: 

• Prioritize your decision making. What problems do you address first, second, 
third...? Explain your first three decisions. 

_Fuel coming from rear of station and flowing down the street 

_Person lying face down 

_Smoke from cornice 

_Fuel pooling in front of station 

_People requesting help/information 

_Lack of resources (foam/EMS) 


Expert group’s response key 


Ranking 

Prioritized Decision-Making 

1 

Person lying face down 

2 

Fuel coming from rear of station and flowing down the street 

3 

Smoke from cornice 


Expert Feedback 

Person lying face down is a known life hazard (visual confirmation) and 
addressing life is a # 1 priority before property. Unit can make quick assessment and 
determine viability of victim removal. The person lying down may possibly be victim of 
robbery and requiring immediate help. 

Fuel coming from rear of station and flowing down the street needs to be 
controlled. If flow cannot be stopped, units must make an attempt to contain or divert 
flow from reaching an ignition source. Leaking fuel is largest threat to life/property, if 
ignited it will intensify an already dynamic and developing situation. 

Extinguishing the fire eliminates a major problem. For example, smoke from the 
fire has potential to expose hospital and apartment building. Investigating the source of 
smoke will reveal fire intensity, determining if a quick stop of fire can be accomplished 
or whether operating personnel should be moved to a safer area 
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Scenario: Earthquake 


4 


Members conducting a perimeter size-up report water under pressure (along with dirt and 
sand) coming from several large cracks in the street that separates the gas station from the 
hospital. This condition also appears to be undermining the sidewalk that leads directly 
into the apartment complex. 


Scenario Decision Point # 4.7: 

• What decisions need to be made regarding street management? List your 
concerns and explain your first three concerns. 


Expert group’s response key: 


Ranking 

Concerns 

1 

Position apparatus out of collapse area. 

2 

Establish safe access / egress from apartment building 

3 

Ensure operating and incoming units are aware of conditions 


Expert Feedback 

Due to the structural integrity of the street, apparatus must be positioned outside 
of a potential street collapse area. The weight of apparatus and already undermined 
street/sink holes are conducive to further collapse. Secondary access points to and from 
hospital/apartment complex also need to be identified. If hospital/apartment buildings 
need to be accessed/evacuated, undermined street/sidewalks will limit availability of safe 
access and egress routes. 

Officers must ensure that both operating and incoming units are aware of 
conditions. Do not assume that everyone knows what you know and see. Individuals 
operate at different situational awareness levels and may not share the same 
comprehension/understanding of the incident as you do. In all likelihood, the stopping of 
unnecessary traffic has already been initiated; however, current conditions may require 
that the isolation zone be expanded, limiting apparatus and excessive pedestrian traffic in 
the area. 
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Scenario: Earthquake 


4 


At approximately 9:30 a.m. you begin to notice the ground shaking in your area. The 
sidewalk and street has now caved in, cutting off your access to the apartment complex 
and hospital. There is a distinct smell of natural gas throughout the area and people are 
beginning to exit the surrounding buildings and hospital, reporting odors of gas/fuel. 


Scenario Decision Point # 4.8: 

• Regarding the hospital and ten-story residential buildings, a decision may 
need to be made concerning evacuating or sheltering in place. List those 
points that must be considered first before any action can be taken. Explain 
your first three points 


Expert group’s response key 


Ranking 

Points to be considered 

1 

Can gas leaks be controlled? 

2 

Available of resources 

3 

Are you evacuating people to a safer location? 


Expert Feedback 

Can units determine if reports of natural gas odors are from high pressure/low 
pressure main? This information will be invaluable to specialized fire units and utility 
companies that have the capability of shutting down leaks and can be useful to these units 
while in route. If gas leaks are within a hospital or surrounding building, can the leaks be 
located, stopped, or controlled? If leak can be stopped, conditions improve (evacuation 
may not be required). If gas leaks are from an exterior source, shelter in place may be 
preferred option 

Availability of resources will dictate what units can and cannot do. Does the 
incident commander have enough resources (fire / civilians) to conduct an evacuation? 
Due to limited resources, can ambulatory patients be moved and non-ambulatory 
sheltered in place? The incident commander must ascertain available resources in the 
form of hospital staff and civilians. 

Regarding the evacuation or sheltering in place of hospital or apartment complex, 
several questions must be addressed. Are apartment and hospital residents in immediate 
danger? Where will you evacuate people and are you exposing them to greater danger 
(street conditions) by evacuating? How safe are access and egress routes from hospital 
and apartment buildings? 
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APPENDIX G 


Experimental group’s decision point questionnaire 

Upon the completion of each scenario, the experimental group completed a 
decision point questionnaire. This questionnaire was designed to capture decision-making 
responses in real time; immediately after they were made. A review of this questionnaire 
revealed the SA / RPD model briefing components received an above-average rating 
regarding usefulness. Although there was no pre-experiment data to compare these 
ratings against, the results are nevertheless encouraging for several reasons. First, the 
experimental group members received a minimal SA / RPD model briefing regarding a 
complex topic, and yet the group’s awareness level was above average. Second, the 
impetus for identifying methods for improving first-responder decision making by raising 
their situation awareness levels is strengthened. The control group did not complete this 
questionnaire 


On a scale of 1-5, with 1 being very low and 5 
being very high, rate the following: 

Tanker 
DP # 1. 

2 

Subway 
DP # 2 .3 

Radiological 

DP #3. 3 

Earthquake 
DP # 4. 6 

How difficult was this particular decisions? 

1 2 3 4 5 

(low) (high) 

2 

3.4 

2.9 

3.1 

How much confidence do you have with the 
decisions you made? 

1 2 3 4 5 

(low) (high) 

4.3 

3.2 

3.9 

3.6 

How much did you rely on experience in your 
decision-making process? (mental models) 

1 2 3 4 5 

(low) (high) 

3.1 

2.9 

2.9 

3.2 

How often did you mentally rehearse a course of 
actions prior to making a decision? (mental 
simulation) 

1 2 3 4 5 

(low) (high) 

4.1 

3.5 

3.6 

3.3 

How successful were you in creating patterns from 
identified cues and using those patterns to 
recognize familiarities and inconsistencies? 

(cues + patterns = decisions) 

1 2 3 4 5 

(low) (high) 

3.7 

3.3 

3.6 

3.5 

Perception: (initial size-up) 

3.6 

3.6 

3.7 

3.9 
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Rate your ability to effectively size up of the 
incident. 

1 2 3 4 5 

(low) (high) 





Comprehension: (making sense of 

surroundings) 

Rate your ability to “make sense of surroundings” 
based on initial size-up, incident goals and 
objectives. 

1 2 3 4 5 

(low) (high) 

4 

3.7 

3.7 

3.9 

Projection: (predicting what will happen next 
and take appropriate actions) 

Rate your ability to predict what will happen next 
and take appropriate actions. 

1 2 3 4 5 

(low) (high) 

3.6 

3.4 

3.5 

3.6 
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APPENDIX H 


Post-experiment questionnaire 

Both the control and experimental groups completed a post-experiment 
questionnaire designed to evaluate scenario difficulty, decision-making confidence, and 
influence of the independent variables. The experimental group rated the influence of 
expert feedback on their overall decision making with a mean rating of 4.1 on a scale of 
1-5 and the helpfulness of the SA/RPD model theory briefing with a mean rating of 3.4. 
These above-average scores, when coupled with the experimental groups’ high scores 
during the actual earthquake scenario, add credibility to research findings, showing that 
expert feedback made a difference. The credibility of research findings are further 
reinforced when compared to the control group that received no feedback and scored 
noticeably lower on the earthquake scenario responses. 

When responding to the post experiment questionnaire, the control group 
members were unaware that the experimental group received any additional treatment. 
Therefore, their ratings were not skewed, biased, or pre-disposed toward additional 
training. Yet, the group rated how influential the receipt of expert feedback would have 
been with a mean rating of 4.5 on a scale from 1-5. The group also rated how helpful a 
pre-scenario decision-making skills/theory training would have been to their decision¬ 
making with a mean rating of 4.5. 
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On a scale of 1-5, rate the preceding exercise 

Regarding decision making, how difficult were these 
scenarios? 

Experimental 

Control 

Renegade Tanker 1 2 3 4 5 

(low) (high) 

3.4 

2.9 

Subway 1 2 3 4 5 

(low) (high) 

3.5 

3.7 

Radiological Attack 1 2 3 4 5 

(low) (high) 

3.9 

3.5 

Earthquake 1 2 3 4 5 

(low) (high) 

4.1 

3.6 

How confident were you with your decision making? 

Experimental 

Control 

Renegade Tanker 1 2 3 4 5 

(low) (high) 

3.7 

3.9 

Subway 1 2 3 4 5 

(low) (high) 

3.4 

3.4 

Radiological Attack 1 2 3 4 5 

(low) (high) 

3.1 

3.1 

Earthquake 1 2 3 4 5 

(low) (high) 

3.1 

3.1 

How influential was the receipt of expert feedback after each 
scenario decision point on subsequent decision making? 

1 2 3 4 5 

(low) (high) 

4.1 


How helpful was the SA/RPD model theory briefing on your 
decision making? (pattern and cue recognition, mental 
models, mental simulation, SA levels-perception, 

comprehension, projection) 

1 2 3 4 5 

(low) (high) 

3.4 


How influential do you believe the receipt of expert feedback 
after each scenario decision point would have been on 
subsequent decision making? 

1 2 3 4 5 

(low) (high) 


4.5 

How helpful do you believe the pre-scenario decision-making 
skills/theory training would have been to your decision 
making? 

1 2 3 4 5 

(low) (high) 


4.5 
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APPENDIX I 


Earthquake scenario score 
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APPENDIX J 


Earthquake Scenario One-way ANOVA 


Dependent (Experiment and Control Combined) 







95% Confidence Interval for 
Mean 




N 

Mean 

Std. 

Deviation 

Std. 

Error 

Lower 

Bound 

Upper Bound 

Minimum 

Maximum 

Experiment 

14 

86.8571 

6.54989 

1.75053 

83.0753 

90.6389 

72.00 

94.00 

Control 

15 

73.6000 

11.59310 

2.99333 

67.1800 

80.0200 

48.00 

100.00 

Total 

29 

80.0000 

11.51397 

2.13809 

75.6203 

84.3797 

48.00 

100.00 


anova Dependent (Experiment and Control Combined) 



Sum of 
Squares 

df 

Mean Square 

F 

Sig. 

Between 

Groups 

1272.686 

1 

1272.686 

14.087 

.001 

Within Groups 

2439.314 

27 

90.345 



Total 

3712.000 

28 
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